[Python-Dev] Pickling instances of nested classes

[Walter Dörwald]

Samuele Pedroni wrote:

>> [...]
>> And having the full name of the class available would certainly help 
>> in debugging.
> 
> that's probably the only plus point but the names would be confusing wrt
>  modules vs. classes.

You'd propably need a different separator in repr. XIST does this:

 >>> from ll.xist.ns import html
 >>> html.a.Attrs.href
<attribute class ll.xist.ns.html:a.Attrs.href at 0x8319284>

> My point was that enabling reduce hooks at the metaclass level has
> propably other interesting applications, is far less complicated than
> your proposal to implement, it does not further complicate the notion of
> what happens at class creation time, and indeed avoids the
> implementation costs (for all python impls) of your proposal and still
> allows fairly generic solutions to the problem at hand because the
> solution can be formulated at the metaclass level.

Pickling classes like objects (i.e. by using the pickling methods in 
their (meta-)classes) solves only the second part of the problem: 
Finding the nested classes in the module on unpickling. The other 
problem is to add additional info to the inner class, which gets pickled 
and makes it findable on unpickling.

> If pickle.py is patched along these lines [*] (strawman impl, not much
> tested but test_pickle.py still passes, needs further work to support
> __reduce_ex__ and cPickle would need similar changes) then this example 
> works:
> 
> 
> class HierarchMeta(type):
>   """metaclass such that inner classes know their outer class, with 
> pickling support"""
>   def __new__(cls, name, bases, dic):
>       sub = [x for x in dic.values() if isinstance(x,HierarchMeta)]

I did something similar to this in XIST, but the problem with this 
approach is that in:

class Foo(Elm):
    pass

class Bar(Elm):
    Baz = Foo

the class Foo will get its _outer_ set to Bar although it shouldn't.

> [...]
>   def __reduce__(cls):
>       if hasattr(cls, '_outer_'):
>           return getattr, (cls._outer_, cls.__name__)
>       else:
>           return cls.__name__

I like this approach: Instead of hardcoding how references to classes 
are pickled (pickle the __name__), deligate it to the metaclass.

BTW, if classes and functions are pickable, why aren't modules:

 >>> import urllib, cPickle
 >>> cPickle.dumps(urllib.URLopener)
'curllib\nURLopener\np1\n.'
 >>> cPickle.dumps(urllib.splitport)
'curllib\nsplitport\np1\n.'
 >>> cPickle.dumps(urllib)
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
   File "/usr/local/lib/python2.4/copy_reg.py", line 69, in _reduce_ex
     raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle module objects

We'd just have to pickle the module name.

Bye,
    Walter Dörwald