[Python-bugs-list] [ python-Bugs-451547 ] pickle / cPickle can't load lambdas
noreply@sourceforge.net
noreply@sourceforge.net
Tue, 29 Oct 2002 14:21:49 -0800
Bugs item #451547, was opened at 2001-08-16 09:33
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=451547&group_id=5470
Category: Python Library
Group: None
Status: Open
Resolution: Fixed
>Priority: 3
Submitted By: Gordon B. McMillan (gmcm)
Assigned to: Guido van Rossum (gvanrossum)
Summary: pickle / cPickle can't load lambdas
Initial Comment:
pickle and cPickle will happily dump a lambda, but on
load, both report:
SystemError: Failed to import class <lambda> from
module __main__
Seen on Py 2.1 & 1.5.2
>>> f = lambda x: x in (1,2,3)
>>> o = cPickle.dumps(f)
>>> f2 = cPickle.loads(o)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
SystemError: Failed to import class <lambda> from
module __main__
>>> o = pickle.dumps(f)
>>> f2 = pickle.loads(o)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "c:\python21\lib\pickle.py", line 951, in loads
return Unpickler(file).load()
File "c:\python21\lib\pickle.py", line 567, in load
dispatch[key](self)
File "c:\python21\lib\pickle.py", line 780, in
load_global
klass = self.find_class(module, name)
File "c:\python21\lib\pickle.py", line 790, in
find_class
raise SystemError, \
SystemError: Failed to import class <lambda> from
module __main__
----------------------------------------------------------------------
>Comment By: Guido van Rossum (gvanrossum)
Date: 2002-10-29 17:21
Message:
Logged In: YES
user_id=6380
Well, the problem is that there are lots of other cases that
the fix also catches. For example:
>>> class C:
class C:
pass
>>> a = C.C()
>>> pickle.dumps(a,1)
I tend to think that catching this is more important than
handling reload() -- since reload() has lots of other
problems like this, it's better to blame reload() and not
try to fix it at the cost of other situations.
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2002-10-28 08:26
Message:
Logged In: YES
user_id=6380
Reopening so I won't forget about this.
----------------------------------------------------------------------
Comment By: Dave Cole (davecole)
Date: 2002-10-25 09:44
Message:
Logged In: YES
user_id=28658
I seem to be suffering from some unintentional consequences
of this fix. I do not think that the pickle should fail in
the follwing case. Wouldn't it be a better idea to just
check for a successful of the pickled class instead of
requiring the imported class to be stored at the same memory
location?
>>> import pickle, copy
>>> o = copy._EmptyClass()
>>> reload(copy)
<module 'copy' from '/usr/lib/python2.2/copy.pyc'>
>>> pickle.dumps(o, 1)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/usr/lib/python2.2/pickle.py", line 978, in dumps
Pickler(file, bin).dump(object)
File "/usr/lib/python2.2/pickle.py", line 115, in dump
self.save(object)
File "/usr/lib/python2.2/pickle.py", line 225, in save
f(self, object)
File "/usr/lib/python2.2/pickle.py", line 477, in save_inst
save(cls)
File "/usr/lib/python2.2/pickle.py", line 225, in save
f(self, object)
File "/usr/lib/python2.2/pickle.py", line 524, in save_global
raise PicklingError(
pickle.PicklingError: Can't pickle <class copy._EmptyClass
at 0x819493c>: it's not the same object as copy._EmptyClass
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2001-08-18 17:23
Message:
Logged In: YES
user_id=6380
I applied Gordon's patch, so this can be closed now.
----------------------------------------------------------------------
Comment By: Gordon B. McMillan (gmcm)
Date: 2001-08-17 17:45
Message:
Logged In: YES
user_id=4923
Patch # 452239 is my attempt to patch cPickle.c to match
Guido's patch to pickle.py.
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2001-08-17 14:51
Message:
Logged In: YES
user_id=6380
I figured it out.
In save_global(), I now try to import the module and then
extract the name from the module; this must give us the same
object. If it doesn't, raise PicklingError.
Checked in as pickle.py rev 1.50. I can't close this yet
because cPickle needs
a similar patch.
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2001-08-16 12:31
Message:
Logged In: YES
user_id=6380
I guess what you're missing is that pickling a function
doesn't pickle the bytecode! It pickles the name instead. So
a reference to "foo.bar" is pickled as "foo.bar", and the
unpickler imports bar from foo.
----------------------------------------------------------------------
Comment By: Gordon B. McMillan (gmcm)
Date: 2001-08-16 12:03
Message:
Logged In: YES
user_id=4923
Hmm. In the (simplistic) case I was trying, I can't see any
significant difference between the lambda and the
equivalent function (names differ, and the func has 2
appended & apparently unreachable bytecodes, but otherwise
the func_* and func_code.co_* attributes match). So what am
I missing?
Lowering priority - I can live without it easily enough.
But if lambda's won't load, they probably shouldn't dump.
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2001-08-16 11:11
Message:
Logged In: YES
user_id=6380
I think I'll have to close this with a won't fix, or "then
don't do that" resolution.
The problem is that whenever a function or class is pickled,
pickle must accept on blind faith that it can also be
unpickled. How would you check that this is indeed the case?
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=451547&group_id=5470