"M.-A. Lemburg" <mal@lemburg.com> writes:
I think that the tuple is not the problem here, it's the fact that so many objects are recorded in the memo to later rebuild recursive structures.
It's not a matter of beliefs: each dictionary entry contributes 12 bytes. Each integer key contributes 12 bytes, each integer position contributes 12 bytes. Each tuple contributes 36 bytes. Assuming pymalloc and the integer allocator, this makes a total of 76 bytes per recorded object. The tuples contribute over 50% to that.
Perhaps pickle could grow an option to assume that a data structure is non-recursive ?! In that case, no data would be written to the memo (or only the id() mapped to 1 to double-check).
That is already possible: You can pass a fake dictionary that records nothing. Regards, Martin