Python Decorator Improvement Idea
Just a small idea that could possibly be useful for python decorators. An idea I had is that it could be possible for a decorator function to declare a parameter which, when the function is called as a decorator, the runtime can fill in various information in the parameters for the decorator to use. Some of the information would be available in all contexts, while other information may only be available in certain contexts.The parameter's value cannot be explicitly specified, defaults to Null except when called as a decorator, and can only be specified once in the function's parameter list. Called as a decorator implies: @decorator def decorated(): pass These are not called as a decorator. The first item's return value would be called as a decorator, thus if it has a decorator parameter, it would be used. @decorator(...) def decorated(): pass def decorated(): pass decorated = decorator(decorated) Any declared callable (function, class instance with __call__, etc) can have an explicitly declared parameter called a decorator information parameter. The syntax could be as follows. "info" is just a name I've chosen and could be any legal name. def decorator(..., @info): pass def wrapper(..., @info): def decorator(obj, @info): ... return obj if info: called directly as decorator, do something else: return decorator Rules: 1. It is not possible for the parameter's value to be directly specified. You can't call fn(info=...) 2. The parameters value is Null except in the cases where it is invoked (the callable called a a decorator). If used in a partial, the decorator parameter would be Null. etc. Information that could be contained in the parameters for all contexts: Variable name Module object declared in Module globals (useful for @export/@public style decorators) Etc Using the decorator in a class context, pass the class object. While the class object hasn't been fully created yet, this could allow accessing attributes of the class (like a registry or such) def decorator(fn, @info): if hasattr(info, "class_obj"): registry = info.class_obj.__dict__.setdefault("_registry", []) registry.append(fn) return fn class MyClass(Base): # Add "method" to the MyClass._registry list @decorator def method(...): pass def call_all(cls): for method in self._registry: method(self) This could also make it possible to use decorators on assignments. Information could be passed to the decorator by the runtime: script_vars = {} def expose(obj, @info): script_vars[info.name] = obj return obj def load(filename): # load script and compile exec(code, script_vars) @expose def script_function(...): pass # This will call the decorator passing in 200 as the object, as well as info.name as the variable being assigned. @expose SCRIPT_CONSTANT = 200 # If stacked, only the first would be used as info.name, in this case SCRIPT_CONSTANT2 @expose SCRIPT_CONSTANT2 = A_VAR = 300 def default(obj, @info): if info.name in info.ctx.vars: return info.ctx.vars[info.name] if info.name in info.globals: return info.globals[info.name] return obj @default X = 12 @default X = 34 print(X) # print 12 since during the second call, X existed and it's value was returned instead of 34, and was assigned to X The two potential benefits I see from this are: 1. The runtime can pass certain information to the decorator, some information in all contexts, and some information in specific contexts such as when decorating a class member, decorating a function defined within another function, etc 2. It would be possible to decorate values directly, as the runtime can pass relevant information such as the variables name This was just an idea I had that may have some use. Thanks, Brian Vanderburg II
On Fri, Jun 15, 2018 at 11:54:42PM -0400, Brian Allen Vanderburg II via Python-ideas wrote:
We can already do this, by writing a decorator factory: @decorator(any parameters you care to pass) def spam(): ... "Explicit is better than implicit" -- it is better to explicitly pass the parameters you want, than to hope that "the runtime" (do you mean the interpreter?) will guess which parameters you need.
Do you mean None? Why do you think it is a good idea to have the same function, the decorator, behave differently when called using decorator syntax and standard function call syntax? To me, that sounds like a terrible idea. What advantage do you see? [...]
That sounds like a recipe for confusion to me. How would you explain this to a beginner? Aside from the confusion that something that looks like a parameter isn't an actual parameter, but something magical, it is also very limiting. It makes it more difficult to use the decorator, since now it only works using @ syntax.
You keep saying Null. What's Null?
The variable name is just the name of the function or class, the first parameter received by the decorator. You can get it with func.__name__. The module globals is already available in globals(). You can either pass it directly as an argument to the decorator, or the decorator can call it itself. (Assuming the decorator is used in the same module it is defined in.) If the decorator is in the same module as the globals you want to access, the decorator can just call globals(). Or use the global keyword. If the decorator is contained in another module, the caller can pass the global namespace as an argument to the decorator: @decorate(globals()) def func(): ... Not the neatest solution in the world, but it works now.
Using the decorator in a class context, pass the class object.
The decorator already receives the class object as the first parameter. Why pass it again?
While the class object hasn't been fully created yet,
What makes you say that?
Writing "hasattr(info, whatever)" is an anti-pattern. By the way, the public interface for accessing objects' __dict__ is to call the vars() function: vars(info.class_obj).set_default(...)
This could also make it possible to use decorators on assignments.
We already can: result = decorator(obj) is equivalent to: @decorator def obj(): ... or @decorator class obj: ... except that we can use the decorator on anything we like, not just a function or class. [...]
That would require a change to syntax, and would have to be a separate discussion. If there were a way to get the left hand side of assignments as a parameter, that feature would be *far* to useful to waste on just decorators. For instance, we could finally do something about: name = namedtuple("name", fields)
No, that would require a second, independent change. We could, if desired, allow decorator syntax like this: @decorate value = 1 but it seems pretty pointless since that's the same as: value = decorator(1) The reason we have @decorator syntax is not to be a second way to call functions, using two lines instead of a single expression, but to avoid having to repeat the name of the function three times: # Repeat the function name three times: def function(): ... function = decorate(function) # Versus only once: @decorate def function(): ... -- Steve
On 06/16/2018 01:22 AM, Steven D'Aprano wrote:
Yes, I meant None instead of Null.
That was just an initial idea. However there would be no reason that the parameter could not be passed directly. Actually if creating one decorator that wraps another decorator, being able to pass the parameter on could be needed. Also, the decorator would still work in normal syntax, only with that parameter set to None
This works with functions and classes but not other values that may not have __name__.
What I mean is used inside the body of a class to decorate a class member: class MyClass(object): @decorator def method(self): pass Using the explicit is better than implicit: class MyClass(object): @decorator(MyClass, ...) def method(self): pass However right now that does not work as MyClass does not exist when the decorator is called. I'm not sure how Python works on this under the hood as it's been a long time since I've looked through the source code. If Python gather's everything under MyClass first before it even begins to create the MyClass object, then it may not be possible, but if Python has already created a class object, and just not yet assigned it to the MyClass name in the module, then perhaps there could be some way to pass that class object to the decorator. I have seen some examples that decorates the class and members to achieve something similar @outerdecorator class MyClass: @decorator def method(self): pass
Agreed it would be a change in syntax. Using the decorator syntax i've mentioned the name being assigned would be passed to that extra info parameter. Python would treat anything in the form of: @decorator NAME = (expression) as a decorator as well: _tmp = (expression) NAME = decorator(_tmp) Right now, there's litlte use as it is just as easy to say directly NAME = decorator(expression) With this idea, it could be possible to do something like this: def NamedTuple(obj @info): return namedtuple(info.name, obj) @NamedTuple Point3 = ["x", "y", "z"]
The two main use cases I had of this idea were basically assignment decorators, pointless as it can just be name = decorator(value), but my idea was to pass to the decorator some metadata such as the name being assigned, and as class member decorators to receive information of the instance of the class object the member is being declared under. A more general idea could be to allow a function call to receive a meta parameter that provides some context information of the call. This parameter is not part of a parameter list, but a special __variable__, or perhaps could be retrieved via a function call. Such contexts could be: 1) Assignment (includes decorators since they are just sugar for name = decorator(name)) The meta attribute assignname would contain the name being assigned to def fn(v): print(__callinfo__.assignname) return v # prints X X = fn(12) # prints MyClass @fn class MyClass: pass # Should assignname receive the left-most assignment result or the rightmost othervar # Perhaps assignname could be a tuple of names being assigned to result = othervar = fn(12) #assignname would be myothervar in this augmented assignment result = [myothervar := fn(12)] # Should expressions be allowed, or would assignname be None? result = 1 + fn(12) With something like this. name = namedtuple("name", ...) could become: def NamedTuple(*args): return namedtuple(__callinfo__.assignname, args) Point2 = NamedTuple("x", "y") Point3 = NamedTuple("x", "y", "z") etc 2) Class context. The a classobj parameter could contain the class object it is called under. This would be a raw object initially as __init__ would not have been called, but would allow the decorator to add attributes to a class def fn(v): print(__callinfo__.classobj) # classobj is None except when the function is called in the body of a class declaration print(__callinfo__.assignname) if __callinfo__.classobj: data = vars(__callinfo__.classobj).setdefault("_registry", {}) data[__callinfo__.assignname] = v return v class MyClass: # print main.MyClass (probably something else since __init__ not yet calls, may just be a bare class object at that timie) # print X # sets MyClass._registry["X"] X = fn(12) # print main.MyClass # print method # sets MyClass._registry["method"] @fn def method(self): pass # print None # print Y Y = fn(12) In this case it's not longer a decorator idea but more of an idea for a called function to be able to retrieve certain meta information about it's call. In the examples above, I used __callinfo__ with attributes, but direct names would work the same: def fn(v): print(__assignname__) # May be None if no assignment/etc if otherfunc(fn(value)) print(__classobj__) # Will be None unless fn is called directly under a class body There may be other contexts and use cases, and better ways. Just an idea.
The idea of having a dunder to introspect the bound variable name has been discussed before. You can find the past discussions in the mailing list archive. If I recall correctly, there were very few use cases beyond namedtuple. With dataclasses available in 3.7, there may be even less interest than before. On Sat, Jun 16, 2018, 9:04 AM Brian Allen Vanderburg II via Python-ideas < python-ideas@python.org> wrote:
On 6/16/2018 8:22 PM, Michael Selik wrote:
One such thread is here: https://mail.python.org/pipermail/python-ideas/2011-March/009250.html Eric
On Fri, Jun 15, 2018 at 11:54:42PM -0400, Brian Allen Vanderburg II via Python-ideas wrote:
We can already do this, by writing a decorator factory: @decorator(any parameters you care to pass) def spam(): ... "Explicit is better than implicit" -- it is better to explicitly pass the parameters you want, than to hope that "the runtime" (do you mean the interpreter?) will guess which parameters you need.
Do you mean None? Why do you think it is a good idea to have the same function, the decorator, behave differently when called using decorator syntax and standard function call syntax? To me, that sounds like a terrible idea. What advantage do you see? [...]
That sounds like a recipe for confusion to me. How would you explain this to a beginner? Aside from the confusion that something that looks like a parameter isn't an actual parameter, but something magical, it is also very limiting. It makes it more difficult to use the decorator, since now it only works using @ syntax.
You keep saying Null. What's Null?
The variable name is just the name of the function or class, the first parameter received by the decorator. You can get it with func.__name__. The module globals is already available in globals(). You can either pass it directly as an argument to the decorator, or the decorator can call it itself. (Assuming the decorator is used in the same module it is defined in.) If the decorator is in the same module as the globals you want to access, the decorator can just call globals(). Or use the global keyword. If the decorator is contained in another module, the caller can pass the global namespace as an argument to the decorator: @decorate(globals()) def func(): ... Not the neatest solution in the world, but it works now.
Using the decorator in a class context, pass the class object.
The decorator already receives the class object as the first parameter. Why pass it again?
While the class object hasn't been fully created yet,
What makes you say that?
Writing "hasattr(info, whatever)" is an anti-pattern. By the way, the public interface for accessing objects' __dict__ is to call the vars() function: vars(info.class_obj).set_default(...)
This could also make it possible to use decorators on assignments.
We already can: result = decorator(obj) is equivalent to: @decorator def obj(): ... or @decorator class obj: ... except that we can use the decorator on anything we like, not just a function or class. [...]
That would require a change to syntax, and would have to be a separate discussion. If there were a way to get the left hand side of assignments as a parameter, that feature would be *far* to useful to waste on just decorators. For instance, we could finally do something about: name = namedtuple("name", fields)
No, that would require a second, independent change. We could, if desired, allow decorator syntax like this: @decorate value = 1 but it seems pretty pointless since that's the same as: value = decorator(1) The reason we have @decorator syntax is not to be a second way to call functions, using two lines instead of a single expression, but to avoid having to repeat the name of the function three times: # Repeat the function name three times: def function(): ... function = decorate(function) # Versus only once: @decorate def function(): ... -- Steve
On 06/16/2018 01:22 AM, Steven D'Aprano wrote:
Yes, I meant None instead of Null.
That was just an initial idea. However there would be no reason that the parameter could not be passed directly. Actually if creating one decorator that wraps another decorator, being able to pass the parameter on could be needed. Also, the decorator would still work in normal syntax, only with that parameter set to None
This works with functions and classes but not other values that may not have __name__.
What I mean is used inside the body of a class to decorate a class member: class MyClass(object): @decorator def method(self): pass Using the explicit is better than implicit: class MyClass(object): @decorator(MyClass, ...) def method(self): pass However right now that does not work as MyClass does not exist when the decorator is called. I'm not sure how Python works on this under the hood as it's been a long time since I've looked through the source code. If Python gather's everything under MyClass first before it even begins to create the MyClass object, then it may not be possible, but if Python has already created a class object, and just not yet assigned it to the MyClass name in the module, then perhaps there could be some way to pass that class object to the decorator. I have seen some examples that decorates the class and members to achieve something similar @outerdecorator class MyClass: @decorator def method(self): pass
Agreed it would be a change in syntax. Using the decorator syntax i've mentioned the name being assigned would be passed to that extra info parameter. Python would treat anything in the form of: @decorator NAME = (expression) as a decorator as well: _tmp = (expression) NAME = decorator(_tmp) Right now, there's litlte use as it is just as easy to say directly NAME = decorator(expression) With this idea, it could be possible to do something like this: def NamedTuple(obj @info): return namedtuple(info.name, obj) @NamedTuple Point3 = ["x", "y", "z"]
The two main use cases I had of this idea were basically assignment decorators, pointless as it can just be name = decorator(value), but my idea was to pass to the decorator some metadata such as the name being assigned, and as class member decorators to receive information of the instance of the class object the member is being declared under. A more general idea could be to allow a function call to receive a meta parameter that provides some context information of the call. This parameter is not part of a parameter list, but a special __variable__, or perhaps could be retrieved via a function call. Such contexts could be: 1) Assignment (includes decorators since they are just sugar for name = decorator(name)) The meta attribute assignname would contain the name being assigned to def fn(v): print(__callinfo__.assignname) return v # prints X X = fn(12) # prints MyClass @fn class MyClass: pass # Should assignname receive the left-most assignment result or the rightmost othervar # Perhaps assignname could be a tuple of names being assigned to result = othervar = fn(12) #assignname would be myothervar in this augmented assignment result = [myothervar := fn(12)] # Should expressions be allowed, or would assignname be None? result = 1 + fn(12) With something like this. name = namedtuple("name", ...) could become: def NamedTuple(*args): return namedtuple(__callinfo__.assignname, args) Point2 = NamedTuple("x", "y") Point3 = NamedTuple("x", "y", "z") etc 2) Class context. The a classobj parameter could contain the class object it is called under. This would be a raw object initially as __init__ would not have been called, but would allow the decorator to add attributes to a class def fn(v): print(__callinfo__.classobj) # classobj is None except when the function is called in the body of a class declaration print(__callinfo__.assignname) if __callinfo__.classobj: data = vars(__callinfo__.classobj).setdefault("_registry", {}) data[__callinfo__.assignname] = v return v class MyClass: # print main.MyClass (probably something else since __init__ not yet calls, may just be a bare class object at that timie) # print X # sets MyClass._registry["X"] X = fn(12) # print main.MyClass # print method # sets MyClass._registry["method"] @fn def method(self): pass # print None # print Y Y = fn(12) In this case it's not longer a decorator idea but more of an idea for a called function to be able to retrieve certain meta information about it's call. In the examples above, I used __callinfo__ with attributes, but direct names would work the same: def fn(v): print(__assignname__) # May be None if no assignment/etc if otherfunc(fn(value)) print(__classobj__) # Will be None unless fn is called directly under a class body There may be other contexts and use cases, and better ways. Just an idea.
The idea of having a dunder to introspect the bound variable name has been discussed before. You can find the past discussions in the mailing list archive. If I recall correctly, there were very few use cases beyond namedtuple. With dataclasses available in 3.7, there may be even less interest than before. On Sat, Jun 16, 2018, 9:04 AM Brian Allen Vanderburg II via Python-ideas < python-ideas@python.org> wrote:
On 6/16/2018 8:22 PM, Michael Selik wrote:
One such thread is here: https://mail.python.org/pipermail/python-ideas/2011-March/009250.html Eric
participants (4)
-
Brian Allen Vanderburg II -
Eric V. Smith -
Michael Selik -
Steven D'Aprano