[Python-ideas] make __closure__ writable
ericsnowcurrently at gmail.com
Fri Mar 23 07:19:29 CET 2012
On Tue, Mar 20, 2012 at 6:06 AM, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
> I did provide such example earlier in this thread. I'm copying and
> pasting it to this mail. Please read the example carefully, as it
> explains why returning new types.FunctionType() is not enough.
> Yes, your approach will work if your decorator is the only one applied.
> But, as I said, if you have many of them (see below), you can't just
> return a new function out of your decorator, you need to change the
> underlying "in-place". Consider the following:
> def modifier(func):
> orig_func = func
> while func.__wrapped__:
> func = func.__wrapped__
> # patch func.__code__ and func.__closure__
> return orig_func # no need to wrap anything
> def some_decorator(func):
> def wrapper(*args, **kwargs):
> # some code
> return func(*args, **kwargs)
> functools.wraps(wrapper, func)
> return wrapper
> def foo():
> # this code needs to be verified/augmented/etc
Couldn't something like the following work?
"""Traverse the decorator "stack" and patch the bottom-most
# relies on __wrapped__ being set at each level of the decorator
stack and on the
# wrapped function being bound in func.__closure__.
if not hasattr(func, "__wrapped__"):
# patch func.__code__ and func.__closure__
code = ...
closure = ...
code = func.__code__
closure = list(func.__closure__)
closure[closure.index(func.__wrapped__)] = modifier(func.__wrapped__)
return type(func)(code, func.__globals__, func.__name__,
Also, I'm guessing that your actual use-case looks more like the following:
from some_third_party_module import foo
#assert foo.__wrapped__ == foo.__closure__
foo = modifier(foo) # hacks up foo.__wrapped__
Hacking the innards of an existing function object is touchy stuff,
probably the riskiest kind of monkey-patching. You're basically
taking the chance of breaking (in ugly, unexpected ways) other code
that uses that function you just hacked. Still, there are certainly
valid use-cases (and we're all consenting adults here).
However, I'd hate for projects to start getting blamed for
difficult-to-debug problems that are the result of some other project
that did this sort of hack. It's nice when your expectations for a
function's behavior (or any code for that matter) can remain stable,
regardless of what libraries are installed along-side.
p.s. I want to reiterate my understanding that nearly everything
involving the internals of functions is pretty delicate (read:
fragile), in part due to being the focal point for optimization.
Hacking it like this is therefore a delicate undertaking and
definitely skirts the edges of creating implementation-specific code.
Don't shy away. Just be extra cautious.
> So, in the above snippet, if you don't want to discard the
> @some_decorator by returning a new function object, you need to modify
> the 'foo' from the @modifier.
> In a complex framework, where you can't guarantee that your magic
> decorator will always be called first, rewriting the __closure__
> attribute is the only way.
> Again, since the __code__ attribute is modifiable, and __closure__
> works in tight conjunction with it, I see no point in protecting it.
> On 2012-03-20, at 5:34 AM, Mark Shannon wrote:
>> Yury Selivanov wrote:
>>> I've created an issue: http://bugs.python.org/issue14369
>> I think that creating an issue may be premature, given that you have had
>> no positive feedback on the idea.
>> I still think making __closure__ mutable is unnecessary.
>> If you insist that it is it, then please provide an example which would
>> work with your proposed change, but cannot be made to work using
>> types.FunctionType() to create a new closure.
>> Python-ideas mailing list
>> Python-ideas at python.org
> Python-ideas mailing list
> Python-ideas at python.org
More information about the Python-ideas