[ python-Bugs-1121475 ] Decorated functions are unpickleable
SourceForge.net
noreply at sourceforge.net
Thu Feb 17 06:10:12 CET 2005
Bugs item #1121475, was opened at 2005-02-12 11:48
Message generated for change (Comment added) made by bcannon
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1121475&group_id=5470
Category: Python Library
Group: Python 2.4
Status: Closed
>Resolution: Wont Fix
Priority: 5
Submitted By: S Joshua Swamidass (sswamida)
Assigned to: Nobody/Anonymous (nobody)
Summary: Decorated functions are unpickleable
Initial Comment:
The decorator feature renders functions impossible to
pickle if you end up wrapping the function in either a
trivial lambda or a callable class. This is a
significant/artificial limitation of the decorator which
should not exist and prevents decorators from being
used for many of my purposes.
==========================
Examples:
==========================
>>> def dec(f):
... return lambda a: f(a)
...
>>> @dec
... def func(a):
... return a
...
>>> import pickle
>>> pickle.dumps(func)
Traceback (most recent call last):
...
pickle.PicklingError: Can't pickle <function <lambda> at
0x40160ae4>: it's not found as __main__.<lambda>
>>> class C:
... def __init__(self, f):
... self.f=f
... def __call__(self, a):
... return f(a)
...
>>> def dec(f):
... return C(f)
>>> @dec
>>> def func(a):
... return a
>>> import pickle
>>> pickle.dumps(func)
Traceback (most recent call last):
.....
pickle.PicklingError: Can't pickle <function func at
0x40160a74>: it's not the same object as __main__.func
==============================
I've found a syntacically ugly ways of working around
the class wrapper bug that don't use decorators.
Perhaps this could be used to create a decorator patch:
==================================
>>> class C:
... def __init__(self, f):
... self.f=f
... def __call__(self, a):
... return f(a)
...
>>> def _func(a):
... return a
>>> func=C(_func)
>>>
>>> import pickle
>>> pickle.dumps(func)
(No error)
----------------------------------------------------------------------
>Comment By: Brett Cannon (bcannon)
Date: 2005-02-16 21:10
Message:
Logged In: YES
user_id=357491
It would be nice if you could pickle decorated functions, but the way
pickle works won't allow it unless the decorator just passes the function
right along:
>>> def dec(func):
... func.meta = "foo"
... return func
...
>>> @dec
... def blah(): pass
...
>>> pickle.dumps(blah)
'c__main__\nblah\np0\n.'
So it is not a hard rule that decorated functions are not pickleable. It all
depends on how the decorator messes with the function it is given.
If you think the documentation for pickle is misleading then please
submit a documentation patch to clarify that decorators could cause
issues if they return the function wrapped.
----------------------------------------------------------------------
Comment By: S Joshua Swamidass (sswamida)
Date: 2005-02-15 17:45
Message:
Logged In: YES
user_id=786138
Yes, everything you said is correct, though i think this is not
ideal behavior. If we decorate a function at the top level of a
module, shouldn't it be pickleable? If not, then as my original
point was, decorated functions are not compatible with pickle.
Is there any case you can give me where a decorated
function is defined in the top level of a module and can be
pickled? I can't think of one, but i could be wrong.
----------------------------------------------------------------------
Comment By: Brett Cannon (bcannon)
Date: 2005-02-15 17:21
Message:
Logged In: YES
user_id=357491
If you read the pickle docs it says you can only pickle "functions defined
at the top level of a module". The lambdas are you having your
decorator return are not defined at the module level and thus do not
meet that requirement.
The same basic issue goes with your callable class. The function you are
storing in self.f is not the same as was is defined at the module level
since self.f is 'func' prior to wrapping while what 'func' has at the global
level is an instance of 'C'.
The reason your example works is because '_func` is defined at the
module level and thus pickle can find it to pickle. Decorators skip the
intermediate step of storing the undecorated function at the module
level.
It would be better to use the pickling protocol to define methods to allow
your callable class to help with the pickling process.
And just so you know, your class examples should be calling self.f
instead of f.
Closing as "won't fix".
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1121475&group_id=5470
More information about the Python-bugs-list
mailing list