performance of pickling & large lists :-(

Martin v. Loewis martin at
Tue Aug 6 15:42:35 EDT 2002

Jeff Epler <jepler at> writes:

> Python's "pickle" will use approximately one dict entry per object, of
> the form
>     { id(obj) : obj }
> this is used to make sure that a structure like 'y' in 
>     y = [[]]*2
> is pickled correctly, and that when reloaded, 'y[0] is y[1]'.
> This accounts for most, if not all, of pickle's memory usage.

Unfortunately, I think this is incorrect: Pickle uses a dictionary of
the form

  { id(obj) : (last_position, obj) }

I think the majority of pickle's memory consumption comes from the

Of course, the last_position field is need so that you know how to
pickle the backwards reference.


More information about the Python-list mailing list