[Python-Dev] The memo of pickle

[Martin v. Loewis]

"M.-A. Lemburg" <mal@lemburg.com> writes:

> I think that the tuple is not the problem here, it's the
> fact that so many objects are recorded in the memo to
> later rebuild recursive structures.

It's not a matter of beliefs: each dictionary entry contributes 12
bytes. Each integer key contributes 12 bytes, each integer position
contributes 12 bytes. Each tuple contributes 36 bytes.

Assuming pymalloc and the integer allocator, this makes a total of 76
bytes per recorded object. The tuples contribute over 50% to that.

> Perhaps pickle could grow an option to assume that a
> data structure is non-recursive ?! In that case, no
> data would be written to the memo (or only the id()
> mapped to 1 to double-check).

That is already possible: You can pass a fake dictionary that records
nothing.

Regards,
Martin