Guido van Rossum wrote:
I think that the tuple is not the problem here, it's the fact that so many objects are recorded in the memo to later rebuild recursive structures.
Now, I believe that recursive structures in pickles are not very common, so the memo is mostly useless in these cases.
Use cPickle, it's much more frugal with the memo, and also has some options to control the memo (read the docs, I forget the details and am in a hurry).
Just to clarify: I don't have a problem with the memo in pickle at all :-) Martin brought up this issue.
Perhaps pickle could grow an option to assume that a data structure is non-recursive ?! In that case, no data would be written to the memo (or only the id() mapped to 1 to double-check).
The memo is also for sharing. There's no recursion in this example, but the sharing may be important:
a = [1,2,3] b = [a,a,a]
Right. I don't think these references are too common in pickles. Zope Corp should know much more about this, I guess, since ZODB is all about pickleing. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/