no pickling code objects?

[holger krekel]

Michael Hudson wrote:
> holger krekel <pyth at devel.trillke.net> writes:
> 
> > just a comment: i need pickling of code objects. Anyone implementing
> > mobile network agents might be interested, also. Up until now
> > i just grab the source/byte code and send it over as a string.
> 
> You can *marhsal* code objects easily enough (hey, that's what marshal
> was invented for!).

But you have to find out which code your data object needs:
the first hurdle.

> But that obviously doesn't solve all the problems:
> 
> > But the basic problem with pickling/unpickling code objects
> > is to transport the transitive closure of code objects: you need to
> > have the code objects which your pickled code objects
> > depends on. And next the code objects which these code
> > objects depend on ...
> 
> Well, indeed.
> 
> This might be another motivation for not pickling functions; the
> decisions get really hairy and basically impossible to guess
> accurately, so Python punts.

Not unless there is a clean-enough solution. I consider passing
a list of modules that are 'resident' or the other way
round a list of modules that are 'movable'. This way
the code pickling could recurse into the code dependencies
related to the data-object and 
raise an error if your data-object needs non-movable code.

If we can solve these issues i would suggest integrating it. 
Consider beeing able to pickle away data together with its
current code. This would reduce versioning problems a lot
because upgrading to a new version would only require pulling the
data out of objects instead of fiddling with the low-level
data structures yourself.

> [...]
> > Has anyone figured these issues out already?
> 
> pyro?

i know. It's marshalling the code every
time an import error occurs on the unpickling side. This
has shortcomings. First you need an established connection
when unpickling. Second if the remote server has an old version
of your code still in sys.modules it will silently be used ...

> But there's a fair chance that your exact problem hasn't been solved
> before and you'll have to work out your exact requirements.  Then you
> can start looking to see if any existing frameworks can get you
> off the ground quicker...

thanks, i am interested in any pointers for previous art,

    holger