[Tutor] problems pickling functions

[Kent Johnson]

Arild B. Næss wrote:
> Hi,
> 
> I'm writing a program for tagging which requires a long time to  
> calculate the parameters. I have therefore tried to write a long  
> program that pickles all the data, and pickles a function that uses  
> these parameters to tag an input sentence.
> 
> But I'm having trouble with loading the function. The odd thing is  
> that it works fine in the interpreter to pickle and load a function:
> 
>  >>> import pickle
>  >>> def simple():
> ...     print "This works"
> ...
>  >>> f=open("simple.txt","w")
>  >>> pickle.dump(simple,f)
>  >>> f.close()
>  >>> s()
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> NameError: name 's' is not defined
>  >>> f=open("simple.txt","r")
>  >>> s=pickle.load(f)
>  >>> s()
> This works
> 
> However when I try to do this with the script simple.py (with the  
> exact same commands as above) it doesn't work:
> 
> $ cat simple.py
> 
> import pickle
> 
> def simple():
>      print "This works"
> 
> f = open("simple.txt","w")
> pickle.dump(simple,f)
> f.close()
> 
> $ python simple.py
> $ python
> Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)
> [GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> import pickle
>  >>> f2 = open("simple.txt","r")
>  >>> s
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> NameError: name 's' is not defined
>  >>> s = pickle.load(f2)
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>    File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ 
> python2.5/pickle.py", line 1370, in load
>      return Unpickler(file).load()
>    File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ 
> python2.5/pickle.py", line 858, in load
>      dispatch[key](self)
>    File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ 
> python2.5/pickle.py", line 1090, in load_global
>      klass = self.find_class(module, name)
>    File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ 
> python2.5/pickle.py", line 1126, in find_class
>      klass = getattr(mod, name)
> AttributeError: 'module' object has no attribute 'simple'
>  >>>
> 
> I don't get this error message, and I'm annoyed – because I'm used to  
> that things that work in the interpreter also work when written as a  
> program.

 From the docs for pickle (13.1.4 What can be pickled and unpickled?):

"Note that functions (built-in and user-defined) are pickled by ``fully 
qualified'' name reference, not by value. This means that only the 
function name is pickled, along with the name of module the function is 
defined in. Neither the function's code, nor any of its function 
attributes are pickled. Thus the defining module must be importable in 
the unpickling environment, and the module must contain the named 
object, otherwise an exception will be raised."

Your code works from the interpreter because 'simple' is still defined. 
If you 'del simple' before you unpickle I think you will get the same 
error as you get in the script.

Searching comp.lang.python for 'pickle function' yields a few possible 
workarounds but they are messy.

Why do you need to pickle the function? Is it created dynamically? Can 
you just pickle the data?

Kent