[Python-Dev] The memo of pickle

[M.-A. Lemburg]

Guido van Rossum wrote:
>>I think that the tuple is not the problem here, it's the
>>fact that so many objects are recorded in the memo to
>>later rebuild recursive structures.
>>
>>Now, I believe that recursive structures in pickles are
>>not very common, so the memo is mostly useless in these
>>cases.
> 
> 
> Use cPickle, it's much more frugal with the memo, and also has some
> options to control the memo (read the docs, I forget the details and
> am in a hurry).

Just to clarify: I don't have a problem with the memo
in pickle at all :-) Martin brought up this issue.

>>Perhaps pickle could grow an option to assume that a
>>data structure is non-recursive ?! In that case, no
>>data would be written to the memo (or only the id()
>>mapped to 1 to double-check).
> 
> The memo is also for sharing.  There's no recursion in this example,
> but the sharing may be important:
> 
> a = [1,2,3]
> b = [a,a,a]

Right. I don't think these references are too common in pickles.
Zope Corp should know much more about this, I guess, since ZODB
is all about pickleing.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/