[Python-bugs-list] [ python-Bugs-494904 ] Cannot pickle a class with a metaclass

[noreply@sourceforge.net]

Bugs item #494904, was opened at 2001-12-18 20:52
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=494904&group_id=5470

Category: Type/class unification
Group: None
Status: Open
Resolution: None
>Priority: 7
Submitted By: Dan Parisien (mathematician)
Assigned to: Guido van Rossum (gvanrossum)
Summary: Cannot pickle a class with a metaclass

Initial Comment:
when pickle retrieves the __reduce__ method of a new 
style class that has a metaclass, instead of 
returning the metaclass's __reduce__ method bound to 
the class, it returns an unbound __reduce__ method of 
that class.

>>> class metaclass(type):
... 	def __reduce__(self):
... 		"""This is metaclass.__reduce__
... 		"""
... 		return type.__reduce__(self)
... 
>>> class newclass(object):
... 	__metaclass__ = metaclass
... 	def __reduce__(self):
... 		"""This is newclass.__reduce__
... 		"""
... 		return object.__reduce__(self)
... 
>>> print newclass.__reduce__.__doc__
This is newclass.__reduce__

when pickle calls object.__reduce__ on newclass, it 
returns an unbound newclass.__reduce__ and not a 
bound metaclass.__reduce__. This has the unfortunate 
side effect of not correctly 'reducing' the class. 
I'm trying to figure out a solution. 



----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-12-18 23:00

Message:
Logged In: YES 
user_id=31435

Sounds like I'd risk it, Guido:  it's not like you could 
break 2.1's pickle behavior for new-style classes with a 
custom metaclass <wink>.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2001-12-18 22:23

Message:
Logged In: YES 
user_id=6380

I've got a patch for pickle.py; a similar patch needs to be
developed for cPickle.c.  I can't find a way to fix this
purely by fixing the type object implementation -- pickle.py
and cPickle.py just don't recognize classes with a custom
metaclass as classes, because of their roots in pre-2.2
patterns. :-(

Should we risk fixing this before 2.2 goes out?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2001-12-18 21:46

Message:
Logged In: YES 
user_id=6380

Very interesting!  Good analysis too (took me a while with
pdb to come to verify it :-). But neither class needs to
define __reduce__ -- the one they inherit from their bases,
type and object, will do the trick just as well.

The problem is that there are two things competing to be
newclass.__reduce__: on the one hand, the unbound method
__reduce__ of newclass instances; on the other hand, the
bound method __reduce__ of newclass, seen as a metaclass
instance. Unfortunately, the unbound method wins, but pickle
is expecting the bound method.

I've tried experimenting with a few ways of fixing this
(e.g. forcing pickle to get the bound __reduce__ method) but
there seem to be powerful forces preventing me from getting
this to work (and I don't mean just a screaming baby :-). I
think maybe the right solution is to make pickle realize
that newclass is a class, and prevent it from pickling it at
all -- it should just pickle a reference to it (i.e. the
module and class name) rather than attempting to pickle its
contents.

Stay tuned.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-12-18 21:19

Message:
Logged In: YES 
user_id=31435

Assigned to Guido.

Dan, leading tabs vanish from the web view, but are visible 
in auto-emailed versions (it's a display issue, it's not 
that the database lost them) -- so don't worry about that.

----------------------------------------------------------------------

Comment By: Dan Parisien (mathematician)
Date: 2001-12-18 21:00

Message:
Logged In: YES 
user_id=118203

it's a shame the tabs do not appear above...
--- Solution

class metaclass(type):
    def __getattribute__(self, name):
        if name in ["__reduce__", "__getstate__", 
"__setstate__"]:
            return lambda s=self, f=getattr(type(self), 
name): f(s)
        return type.__getattribute__(self, name)

this fixed my bug, but it may not work for everybody. My 
suggestion is if you are to pickle a new style class, you 
should call 
type(new_style_class).__reduce__(new_style_class) instead 
of new_style_class.__reduce__()


----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=494904&group_id=5470