Is there anything that pickle + copy_reg cannot serialize?
Jean-Paul Calderone
exarkun at divmod.com
Thu Dec 8 11:26:18 EST 2005
On Thu, 08 Dec 2005 22:42:32 +0800, Maurice LING <mauriceling at acm.org> wrote:
>Hi,
>
>I need to look into serialization for python objects, including codes,
>recursive types etc etc. Currently, I have no idea exactly what needs to
>be serialized, so my scope is to be as wide as possible.
>
>I understand that marshal is extended by pickle to serialize class
>instances, shared elements, and recursive data structures
>(http://www.effbot.org/librarybook/pickle.htm) but cannot handle code
>types. pickle can be used together with copy_reg and marshal to
>serialize code types as well
>(http://www.effbot.org/librarybook/copy-reg.htm).
>
>So my question will be, are there anything that pickle/copy_reg/marshal
>combination cannot serialize? If so, what are the workarounds?
Since copy_reg lets you specify arbitrary code to serialize arbitrary
objects, you shouldn't run into any single object that you cannot
serialize to a pickle.
However, both pickle implementations are recursive, so you will be
limited by the amount of memory you can allocate for your stack. By
default, this will limit you to something like object graphs 333 edges
deep or so (if I'm counting stack frames correctly). Note that this
does not mean you cannot serialize more than 333 objects at a time,
merely that if it takes 333 or more steps to go from the first object
to any other object in the graph (using the traversal order pickle
uses), the pickling will fail. You can raise this limit, to a point,
with sys.setrecursionlimit().
Jean-Paul
More information about the Python-list
mailing list