pickling instances of metaclass generated classes

[lars van gemerden]

On Dec 29, 8:55 pm, Ian Kelly <ian.g.ke... at gmail.com> wrote:
> On Thu, Dec 29, 2011 at 2:55 AM, lars van gemerden <l... at rational-it.com> wrote:
>
> > Hello,
>
> > Can someone help me with the following:
>
> > I am using metaclasses to make classes and these classes to make
> > instances. Now I want to use multiprocessing, which needs to pickle
> > these instances.
>
> > Pickle cannot find the class definitions of the instances. I am trying
> > to add a line to the __new__ of the metaclass to add the new class
> > under the right name in the right module/place, so pickle can find
> > it.
>
> > Is this the right approach? Can anyone explain to me where/how to add
> > these classes for pickle to find and maybe why?
>
> It sounds like you're trying to do something like this?
>
> >>> class MetaClass(type):
>
> ...     pass
> ...>>> instance = MetaClass('<Anonymous>', (object,), {})()
> >>> instance
>
> <__main__.<Anonymous> object at 0x00CC00F0>>>> import pickle
> >>> pickle.dumps(instance)
>
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "c:\python27\lib\pickle.py", line 1374, in dumps
>     Pickler(file, protocol).dump(obj)
>   File "c:\python27\lib\pickle.py", line 224, in dump
>     self.save(obj)
>   File "c:\python27\lib\pickle.py", line 331, in save
>     self.save_reduce(obj=obj, *rv)
>   File "c:\python27\lib\pickle.py", line 401, in save_reduce
>     save(args)
>   File "c:\python27\lib\pickle.py", line 286, in save
>     f(self, obj) # Call unbound method with explicit self
>   File "c:\python27\lib\pickle.py", line 562, in save_tuple
>     save(element)
>   File "c:\python27\lib\pickle.py", line 295, in save
>     self.save_global(obj)
>   File "c:\python27\lib\pickle.py", line 748, in save_global
>     (obj, module, name))
> pickle.PicklingError: Can't pickle <class '__main__.<Anonymous>'>:
> it's not found as __main__.<Anonymous>
>
> Yeah, pickle's not going to work with anonymous classes.  As you
> suggest, you could dynamically add the classes to the module namespace
> so that pickle.dumps will find them, but bear in mind that they will
> also have to exist when calling pickle.loads, so you will need to be
> able to reconstruct the same anonymous classes before unpickling later
> on.
>
> Cheers,
> Ian

Thank you Ian for the minimal example. This is almost the case i was
trying to discribe, however the classes will be named at runtime and
the class definitions will be persisted in a database.

Can you help me with how to add the classes to the correct namespace?
The application spans multiple modules (or compared to the example,
the metaclass definition will be in another module then one where the
class and instance will be generated).

Cheers, Lars