Serializing functions

[Robert Kern]

On 6/17/10 8:23 AM, Matteo Landi wrote:
> Some weeks ago, here on the mailing list I read about picloud[1], a
> python library used for cloud-computing; I was impressed by its
> simplicity, here is an example:
>
>>>> import cloud
>>>> def square(x):
> ...  return x * x
>>>> cloud.call(square, 10)
>>>> cloud.result()
> 100
>
> So, I tried to figure out how to achieve the same result, i.e. define a
> local generic function and send it somewhere for remote execution, and
> the get the result back.
> So I thought about serialization (pickle): I made a few tries and it
> seemed to work.. but I was wrong, because I never tried to load pickled
> data from another session different from the one used to pickle data
> itself. If you try and pickle a function, it is not pickled as a whole,
> indeed, once you unpickle it, it will raise an exception telling you
> that the target function was not found in the current module.
>
> So I'm here, with nothing in my hands; how would you implement this?

PiCloud's client library is LGPLed. Go take a look for yourself. It's pretty nifty.

[Disclosure: My employer, Enthought, has partnered with PiCloud to provide our 
Enthought Python Distribution to PiCloud users. However, I say that it's nifty 
purely because it is.]

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco