Object serialization: transfer from a to b (non-implemented code on b)

[Gabriel Rossetti]

Hello everyone,

I am trying to serialize a function, class, etc and transfer it, have it 
unserialized and used. The thing is that this code is not defined on the 
receiving side and thus it does not work. I tried the following tests:

Terminal A:

 >>> import pickle
 >>> def test(): pass
...
 >>> pickle.dumps(test)
'c__main__\ntest\np0\n.'
 >>>

Terminal B:

 >>> import pickle
 >>> pickle.loads('c__main__\ntest\np0\n.')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/pickle.py", line 1374, in loads
    return Unpickler(file).load()
  File "/usr/lib/python2.5/pickle.py", line 858, in load
    dispatch[key](self)
  File "/usr/lib/python2.5/pickle.py", line 1090, in load_global
    klass = self.find_class(module, name)
  File "/usr/lib/python2.5/pickle.py", line 1126, in find_class
    klass = getattr(mod, name)
AttributeError: 'module' object has no attribute 'test'


Ok, so pickle needs the code to be defined on both sides, so I tried 
marshal:

Terminal A:

 >>> import marshal
 >>> marshal.dumps(test)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: unmarshallable object

Ok, as the docs say, marshal is quite limited, no functions or user 
classes can be marshalled, I did get it to work like this though :

 >>> import inspect
 >>> marshal.dumps(inspect.getmembers(test, inspect.iscode)[0][1])
'c\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00C\x00\x00\x00s\x04\x00\x00\x00d\x00\x00S(\x01\x00\x00\x00N(\x00\x00\x00\x00(\x00\x00\x00\x00(\x00\x00\x00\x00(\x00\x00\x00\x00s\x07\x00\x00\x00<stdin>t\x04\x00\x00\x00test\x01\x00\x00\x00s\x00\x00\x00\x00'

ok, but can I unmarshal it?

Terminal B:

 >>> 
marshal.loads('c\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00C\x00\x00\x00s\x04\x00\x00\x00d\x00\x00S(\x01\x00\x00\x00N(\x00\x00\x00\x00(\x00\x00\x00\x00(\x00\x00\x00\x00(\x00\x00\x00\x00s\x07\x00\x00\x00<stdin>t\x04\x00\x00\x00test\x01\x00\x00\x00s\x00\x00\x00\x00')
<code object test at 0xb7591578, file "<stdin>", line 1>

ok, it seams to work...

 >>> 
marshal.loads('c\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00C\x00\x00\x00s\x04\x00\x00\x00d\x00\x00S(\x01\x00\x00\x00N(\x00\x00\x00\x00(\x00\x00\x00\x00(\x00\x00\x00\x00(\x00\x00\x00\x00s\x07\x00\x00\x00<stdin>t\x04\x00\x00\x00test\x01\x00\x00\x00s\x00\x00\x00\x00')()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'code' object is not callable

ok, logical, it's a code object...

 >>> 
eval(marshal.loads('c\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00C\x00\x00\x00s\x04\x00\x00\x00d\x00\x00S(\x01\x00\x00\x00N(\x00\x00\x00\x00(\x00\x00\x00\x00(\x00\x00\x00\x00(\x00\x00\x00\x00s\x07\x00\x00\x00<stdin>t\x04\x00\x00\x00test\x01\x00\x00\x00s\x00\x00\x00\x00'))

ok, this works, not super pretty, but it works

now user objects?

 >>> class A(object): pass
...
 >>> a = A()
 >>> marshal.dumps(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: unmarshallable object

ok, expected...

 >>> inspect.getmembers(a, inspect.iscode)
[]

ok, not cool, can't use the previous method...


Ok, now a word about what I'm trying to do, I'm trying to run python 
code in a new process, like multiprocessing does, but without 
multiprocessing. I'm trying to serialise the code because I 'd like to 
do something like:

pseudo-code, serialize/unserialize/process are made up:

def newProcess(func, *args, **kwargs):
    func = serialize(func)
    args = serialize(args)
    kwargs = serialize(kwargs)
    process("python -c 'import serialize, unserialize;print 
serialize(unserialize(\'%s\')(*unserialize(\'%s\'), 
**unserialize(\'%s\')))'" % (func, args, kwargs))
   #read result from stdout

I'm greatly simplifying this because I'm using a framework, but that's 
the basic idea.

thanks,
Gabriel