problem doing unpickle in an exec statement

Peter Otten __peter__ at web.de
Thu Jul 24 03:13:59 EDT 2008


Danny Shevitz wrote:

> Howdy,
> 
> In my app I need to exec user text that defines a function. I want this
> function to unpickle an object. Pickle breaks because it is looking for
> the object definition that isn't in the calling namespace.
> 
> I have mocked up a simple example that shows the problem. Run this
> first code (from create_pickle.py) to create the pickle.
> 
> create_pickle.py: (run this first)
> 
> #############################################
> import cPickle
> 
> # the pickle file name
> file_name = 'd:\\temp\\test1.pickle'
> 
> # define a class
> class Tree(object):
> pass
> 
> 
> def main():
> # instantiate
> t = Tree()
> 
> # create the sweet pickle
> fp = open(file_name, 'wb')
> cPickle.dump(t, fp)
> fp.close()
> 
> # try to unpickle directly
> fp = open(file_name, 'rb')
> result = cPickle.load(fp)
> fp.close()
> print "unpickling directly works just fine, result = ", result
> 
> if __name__=='__main__':
> main()
> #############################################
> 
> run this second:
> 
> exec_pickle.py
> #############################################
> # this file shows a problem with sweet pickle in an exec statement
> 
> # the pickle file name
> file_name = 'd:\\temp\\test1.pickle'
> 
> # code to be turned into a function
> code_text = '''
> def include():
>   print "this works!"
> '''
> 
> # a function for creating functions
> def create_fun(code_text):
> clean_dict = {}
> exec code_text in clean_dict
> return clean_dict['include']
> 
> # include_fun is a bona fide function
> include_fun = create_fun(code_text)
> 
> # this works
> include_fun()
> 
> 
> # now try to load the pickle in an exec statement
> code_text = '''
> def include(file_name):
>   print "processing file_name: ", file_name
>   import cPickle
>   fp = open(file_name, "rb")
>   result = cPickle.load(fp)
>   fp.close()
>   print "result = ", result
> '''
> 
> # create the new include_fun
> include_fun = create_fun(code_text)
> 
> # run it
> include_fun(file_name)
> 
> #############################################
> 
> Can anyone enlighten me what I need to do to exec_pickle.py
> to get this to work?

Pickling saves the name of the module and the class (and of course the
instance data). Because you put the class in your main script the module
name is __main__, and when you unpickle later pickle imports __main__ which
unfortunately is now a different script that doesn't contain the/a Tree
class. To solve that problem put the Tree class in a separate module, say
tree.py, that can be imported by both create_pickle.py and exec_pickle.py.
You only need an explicit 

from tree import Tree

in create_pickle.

Peter





More information about the Python-list mailing list