Notes on @ram.cache

Just wanted to highlight this message from the dev list...  Basically, caching a list of catalog brains using @ram.cache from plone.memoize is a no-no.  Here there be dragons...
David

Begin forwarded message:

From: Hanno Schlichting <plone@hannosch.info>
Date: July 17, 2008 1:14:23 AM PDT
To: plone-developers@lists.sourceforge.net
Subject: Re: [Plone-developers] memoize and ConnectionStateErrors

Hi.

David Glick wrote:
Is there any reason RAM caching using plone.memoize's @ram.cache 
decorator might result in sporadic ConnectionStateErrors?  I've been 
experiencing these lately, and they appeared around the same time I 
added memoization, so I'm suspicious.  (The methods I'm caching are 
returning a list of catalog brains, FWIW.)  If anyone knows a possible 
reason, or can give an explanation of what leads to this error in 
general, I'd be most grateful :)

From the docstring of zope.app.cache.ram:

The idea behind the `RAMCache` is that it should be shared between
threads, so that the same objects are not cached in each thread. This is
achieved by storing the cache data structure itself as a module level
variable (`RAMCache.caches`). This, of course, requires locking on
modifications of that data structure.

Now what does this tell you? Do not store objects in a RAM cache that
belong to a ZODB connection. Any peristent object including wrappers
around those, like catalog brains are a definitive no-no!

Often you won't see the connection state errors at first, since the
objects are all still in the thread local ZODB cache of all threads. Try
lowering your cache-size from the default 5000 to something like 100 and
see what you get. This should warn you a lot earlier (i.e. during
development) when you are doing something wonky ;)

Hanno