[Python-bugs-list] [ python-Bugs-494904 ] Cannot pickle a class with a metaclass
noreply@sourceforge.net
noreply@sourceforge.net
Wed, 19 Dec 2001 08:58:11 -0800
Bugs item #494904, was opened at 2001-12-18 20:52
You can respond by visiting:
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=494904&group_id=5470
Category: Type/class unification
>Group: Python 2.2
>Status: Closed
>Resolution: Fixed
Priority: 7
Submitted By: Dan Parisien (mathematician)
Assigned to: Guido van Rossum (gvanrossum)
Summary: Cannot pickle a class with a metaclass
Initial Comment:
when pickle retrieves the __reduce__ method of a new
style class that has a metaclass, instead of
returning the metaclass's __reduce__ method bound to
the class, it returns an unbound __reduce__ method of
that class.
>>> class metaclass(type):
... def __reduce__(self):
... """This is metaclass.__reduce__
... """
... return type.__reduce__(self)
...
>>> class newclass(object):
... __metaclass__ = metaclass
... def __reduce__(self):
... """This is newclass.__reduce__
... """
... return object.__reduce__(self)
...
>>> print newclass.__reduce__.__doc__
This is newclass.__reduce__
when pickle calls object.__reduce__ on newclass, it
returns an unbound newclass.__reduce__ and not a
bound metaclass.__reduce__. This has the unfortunate
side effect of not correctly 'reducing' the class.
I'm trying to figure out a solution.
----------------------------------------------------------------------
>Comment By: Guido van Rossum (gvanrossum)
Date: 2001-12-19 08:58
Message:
Logged In: YES
user_id=6380
Thanks for reporting! Fixed in CVS, with a simpler version
of the patch below; also for cPickle. Adding tests too...
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2001-12-18 23:00
Message:
Logged In: YES
user_id=31435
Sounds like I'd risk it, Guido: it's not like you could
break 2.1's pickle behavior for new-style classes with a
custom metaclass <wink>.
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2001-12-18 22:23
Message:
Logged In: YES
user_id=6380
I've got a patch for pickle.py; a similar patch needs to be
developed for cPickle.c. I can't find a way to fix this
purely by fixing the type object implementation -- pickle.py
and cPickle.py just don't recognize classes with a custom
metaclass as classes, because of their roots in pre-2.2
patterns. :-(
Should we risk fixing this before 2.2 goes out?
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2001-12-18 21:46
Message:
Logged In: YES
user_id=6380
Very interesting! Good analysis too (took me a while with
pdb to come to verify it :-). But neither class needs to
define __reduce__ -- the one they inherit from their bases,
type and object, will do the trick just as well.
The problem is that there are two things competing to be
newclass.__reduce__: on the one hand, the unbound method
__reduce__ of newclass instances; on the other hand, the
bound method __reduce__ of newclass, seen as a metaclass
instance. Unfortunately, the unbound method wins, but pickle
is expecting the bound method.
I've tried experimenting with a few ways of fixing this
(e.g. forcing pickle to get the bound __reduce__ method) but
there seem to be powerful forces preventing me from getting
this to work (and I don't mean just a screaming baby :-). I
think maybe the right solution is to make pickle realize
that newclass is a class, and prevent it from pickling it at
all -- it should just pickle a reference to it (i.e. the
module and class name) rather than attempting to pickle its
contents.
Stay tuned.
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2001-12-18 21:19
Message:
Logged In: YES
user_id=31435
Assigned to Guido.
Dan, leading tabs vanish from the web view, but are visible
in auto-emailed versions (it's a display issue, it's not
that the database lost them) -- so don't worry about that.
----------------------------------------------------------------------
Comment By: Dan Parisien (mathematician)
Date: 2001-12-18 21:00
Message:
Logged In: YES
user_id=118203
it's a shame the tabs do not appear above...
--- Solution
class metaclass(type):
def __getattribute__(self, name):
if name in ["__reduce__", "__getstate__",
"__setstate__"]:
return lambda s=self, f=getattr(type(self),
name): f(s)
return type.__getattribute__(self, name)
this fixed my bug, but it may not work for everybody. My
suggestion is if you are to pickle a new style class, you
should call
type(new_style_class).__reduce__(new_style_class) instead
of new_style_class.__reduce__()
----------------------------------------------------------------------
You can respond by visiting:
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=494904&group_id=5470