Pickling classes (not class instances)

Nicolas M. Thiéry Nicolas.Thiery at u-psud.fr
Sat Feb 14 03:19:53 EST 2009


         Dear python developers,

I got no answer to my previous post on this thread "Pickling classes
(not class instances)".
This issue is a show stopper for our project. Any suggestion for where
to ask?

Thanks in advance!

Best regards,
					Nicolas

> Purpose of this e-mail:
> -----------------------
>
> How to customize how a class (not an instance of a class!!!) is
> pickled?
>
> Example:
> ==============================================================================
> class metaclass(type):
>     def __new__(mcs, name, bases, dict):
>         print " running metaclass.__new__"
>         return type.__new__(mcs, name, bases, dict)
>     #
>     def __reduce_class__(self):
>         print "reducing a class"
>         # do the real work
>
> c = metaclass("foo", (object,), dict())
>
> import copy_reg
> copy_reg.pickle(metaclass, metaclass.__reduce_class__)
>
> pickle.dumps(c)
> ---------------------------------------------------------------------------
> PicklingError                             Traceback (most recent call
> last)
> ...
> PicklingError: Can't pickle <class '__main__.foo'>: it's not found as
> __main__.foo
> ==============================================================================
>
> Context:
> --------
>
> I am working on the Sage project (www.sagemath.org), and more
> precisely on the category infrastructure. The code is building lots of
> classes on the fly by composing preexisting classes by inheritance
> (for the curious, see the doc of the class Category inhttp://sage.math.washington.edu:2144/file/1567cea09170/categories-nt....).
>
> Since those classes are built on the fly, they cannot be pickled with
> the default mechanism of name lookup. A proper pickling would be to
> rebuild the class anew. Nothing problematic, except for the question
> above.
>
> Discussion:
> -----------
>
> It sounds like copy_reg would be the natural way to go (as in the
> example above). However, its documentation suggests that it explicitly
> is not intended for pickling classes, e.g. first paragraph of:
>
> http://docs.python.org/library/copy_reg.html#module-copy_reg
>
> is:
>
>   The copy_reg module provides support for the pickle and cPickle
>   modules. The copy module is likely to use this in the future as
>   well.  It provides configuration information about object
>   constructors which are not classes. Such constructors may be factory
>   functions or class instances.
>
> And indeed, looking up at the code of pickle (l. 289-299 of pickle.py)
> (and similarly in cpickle), the copy-reg dispatch is explicit bypassed
> for metaclasses:
>
>         # Check for a class with a custom metaclass; treat as regular
> class
>         try:
>             issc = issubclass(t, TypeType)
>         except TypeError: # t is not a class (old Boost; see SF
> #502085)
>             issc = 0
>         if issc:
>             self.save_global(obj)
>             return
>
>         # Check copy_reg.dispatch_table
>         reduce = dispatch_table.get(t)
>
> Which causes the failure above.
>
> Is there a specific reason for this restriction?
>
> Would it be thinkable to move up the copy reg dispatch before the
> metaclass treatment in pickle and cPickle? I did it locally, and it
> fixed my problem.
>
> If not, what would be the right way to achieve this?
>
> Many thanks in advance!
>
> Best regards,
>                                 Nicolas
> --
> Nicolas M. Thiéry "Isil" <nthi... at users.sf.net>http://Nicolas.Thiery.name/




More information about the Python-list mailing list