pickling instances of metaclass generated classes

[Ian Kelly]

On Thu, Dec 29, 2011 at 2:55 AM, lars van gemerden <lars at rational-it.com> wrote:
> Hello,
>
> Can someone help me with the following:
>
> I am using metaclasses to make classes and these classes to make
> instances. Now I want to use multiprocessing, which needs to pickle
> these instances.
>
> Pickle cannot find the class definitions of the instances. I am trying
> to add a line to the __new__ of the metaclass to add the new class
> under the right name in the right module/place, so pickle can find
> it.
>
> Is this the right approach? Can anyone explain to me where/how to add
> these classes for pickle to find and maybe why?

It sounds like you're trying to do something like this?

>>> class MetaClass(type):
...     pass
...
>>> instance = MetaClass('<Anonymous>', (object,), {})()
>>> instance
<__main__.<Anonymous> object at 0x00CC00F0>
>>> import pickle
>>> pickle.dumps(instance)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "c:\python27\lib\pickle.py", line 1374, in dumps
    Pickler(file, protocol).dump(obj)
  File "c:\python27\lib\pickle.py", line 224, in dump
    self.save(obj)
  File "c:\python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "c:\python27\lib\pickle.py", line 401, in save_reduce
    save(args)
  File "c:\python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "c:\python27\lib\pickle.py", line 562, in save_tuple
    save(element)
  File "c:\python27\lib\pickle.py", line 295, in save
    self.save_global(obj)
  File "c:\python27\lib\pickle.py", line 748, in save_global
    (obj, module, name))
pickle.PicklingError: Can't pickle <class '__main__.<Anonymous>'>:
it's not found as __main__.<Anonymous>


Yeah, pickle's not going to work with anonymous classes.  As you
suggest, you could dynamically add the classes to the module namespace
so that pickle.dumps will find them, but bear in mind that they will
also have to exist when calling pickle.loads, so you will need to be
able to reconstruct the same anonymous classes before unpickling later
on.

Cheers,
Ian