A __decoration_call__ method for Callable objects (WAS: Decorators on variables)

On Wed, May 26, 2021 at 7:54 AM Steven D'Aprano <steve@pearwood.info> wrote:
....and previously Steve also said: On Tue, May 25, 2021 at 3:28 AM Steven D'Aprano <steve@pearwood.info> wrote:
Well, in actuality even if it were implemented the way you described, it is still going to be very different--- I might even say radically different. These two ideas of a decorator syntax result are not the same: RESULT A: function decorator # func = decorator("spam")(func) RESULT B: variable decorator # name = decorator("spam")("name") ...because func is passed as an object, but "name" a string representing the name of the object. Two very different things. For this reason I think I would agree even more so that the differences in the decorator behavior would be an extremely significant point of confusion. This got me to thinking: what if access to the variable name were provided by another means, and ONLY when the decorator syntax is employed? So then I started to write this big long email and it kind of got out of hand. Hopefully it isn't a disaster. FIRST LAYER: RICK'S (LIKELY HALF-BAKED) COUNTER PROPOSAL Maybe employment of decorator syntax could OPTIONALLY trigger a new dunder method-- here I'll just call it __decoration_call__-- with the signature: def __decoration_call__(self, obj: Any, by_name: str) -> Any: ... Before I describe what I intend by this, first I will stipulated that what I am proposing here should only be implemented so that the behavior of all currently existing decorators (i.e., all callables) would remain exactly as it does today. So, for any existing callable with the name decorator, this: @decorator("spam this") def func(): ... ...continues to mean this, just as it does today: def func(): ... func = decorator("spam this")(func) My idea is to optionally allow any callable object to write a __decoration_call__ method that gets called in lieu of the __call__ method when the callable object is employed using decorator syntax. When this happens, the decorated named is supplied- not counting self- as the first argument (e.g., by_name), which contains the str value of the name the decorator was applied to. Let's explain using code examples. In actuality, unless I'm wrong (I might be; not an expert) current decorator syntax is really sugar for: def func(): ... func = decorator.__call__("spam this").__call__(func) My proposal is to make it such that: @decorator def func(): ... ...*can result* in this: def func(): ... func = decorator.__decoration_call__( func, "func") And also so that this: @decorator("spam this") def func(): ... ...*can result* in this: def func(): ... func = decorator.__call__("spam this").__decoration_call__(func, "func") I say "*can result*" because this has the following limitations: 1. occurs only when the callable object is employed as a decorator 2. occurs only when __decoration_call__ exists (and if it doesn't, just revert back to the usual __call__(func_object, obj)) Here is an example to further illustrate the suggested behavior. It is not intended to demonstrate a useful example. You could write a callable object like this: class Decorator: def __call__(self, obj): print("spam") return obj def __decoration_call__(self , obj, by_name): print("eggs") print(by_name) return obj decorator = Decorator() And the behavior would be like this: def func(): ... func = decorator(func) # Console output: # 'spam' @decorator def func(): ... # Console output: # 'eggs' # 'func' In order to preserve current behavior, the built-in <class 'function'> type would not grow its own new __decoration_call__ method. So in the case of existing normal functions, and also in the case of existing custom Callable objects, the fallback would be to just use __call__ as normal. In this way, all of the proposal is optional. But as illustrated above, the new dunder can be implemented by any customized Callable class. I think __decoration_call__ by itself could provide a lot of interesting possibilities for existing class and function decoration. But my main motivation for this is as a different way of implementing the variable decorator idea proposed by Jeremiah. SECOND LAYER: VARIABLE DECORATORS With __decoration_call__ in place, we could do a lot of interesting things if we take the additional step of broadening existing decorator syntax such that you can decorate not just functions and classes but also variables (similar the variable decorator proposal recently proposed by Jeremiah). But first let's define the variable decorator syntax behavior. Using the same decorator object I have defined above, and borrowing Jeremiah's original example, the new syntax would transform this: @decorator variable ...into this: variable = decorator.__decoration_call__(None, "variable") EXPLANATION OF None: in the example above there is no assignment on the decorated line, and so there is no object to pass to the dunder at that point . Because of this, the first argument, which is usually the object being decorated, is supplied as None in the call to __decoration_call__ . Only the name being assigned to the as-yet uncreated object exists. And the new syntax would transform this: @decorator variable = "spam" ...into this: variable = decorator.__decoration_call__(variable, "variable") With the behavior of this variable decoration defined as shown above, here are some of the interesting possibilities I was considering. One possibility is avoiding the pitfalls of name repetition in the RAD framework context previously mentioned by Stéfane Fermigier in Jeremiah's variable decorator thread (see his post for examples). It could see other frameworks, like web frameworks, making use if this too. (Here I will repeat some of the possibilities suggested by Jeremiah in his first post) The standard library and other library functions could, as needed, make use of __decoration_call__ as they see fit. I could see factories like namedtuple, make_dataclass, several typing factories, and Enum, implementing __decoration_call__ so you can say things like: @namedtuple Point = "x y z" @make_dataclass Point = [("x", int), ("y", int), ("z", int)] @typing.TypeVar T = (str, bytes) @typing.NewType UserId = int @enum.Enum Colors = "RED GREEN BLUE" Admittedly, these examples are not all that compelling on their own, but I think they read really well and they would allow you to avoid retyping a name (and potentially making a mistake in retyping it), so I think I really like it. ___ SIDEBAR I also note that the above examples use a different idiom than suggested by Jeremiah. He has suggested these: @namedtuple("x y z") Point @make_dataclass([("x", int), ("y", int), ("z", int)]) Point @typing.TypeVar( (str, bytes) ) T @typing.NewType(int) UserId @enum.Enum("RED GREEN BLUE") Colors ...but using my proposal I do not see a way to make this idiom easily work with the existing factories. I am not sure which of these two idioms people would prefer. I can see pros and cons for both idioms. If people preferred the second idiom, new decorator objects could be added for it. ___ One interesting possibility might be to modify inspect.getsource so that it can retrieve the RHS of the line on which it appears*: @inspect.getsource my_var = 1/2 print(my_var) # Result # '@inspect.getsource my_var = 1/2' * NOTE: I am admittedly hand-waving this. Maybe there are reasons getsource would not be able to do this? Using the rhs string from this new version of getsource, it would be a pretty simple matter for a symbolic math library like sympy to replace the float value resulting from 1/2 in my_var with a symbolic math value: @sympy.sympify my_var = 1/2 # Result is not 0.5, but: # 1/2 ...or create a Fraction (using a modified Fraction type): @Fraction my_var = 1/2 # Result is not 0.5, but: # Fraction(1, 2) ..or a Decimal (using a modified Decimal type): @Decimal my_var = 0.5 # Result is not 0.5, but: # Decimal('0.5') MAYBE A DEFAULT DUNDER AFTER ALL...? One final possibility: I have said above we would not supply any default __decoration_call__ method, and instead fallback on __call__. But it might be beneficial to supply a default __decoration_call__ with the following implementation: def __decoration_call__(self, func, by_name): if func is None: return self(by_name) return self(func) This would allow things like this, "out of the box": @typing.NewType Color = str @Color GREEN ...which becomes: Color = typing.NewType.__decoration_call__(str, "Color") GREEN = Color.__decoration_call__(None, "GREEN") In the above, Color is just a stand-in for str. If there is a default __decoration_call__ method as I wrote above, then: Color.__decoration_call__(None, "GREEN") ...is actually just: str.__decoration_call__(None, "GREEN") ..and the default implementation just returns: str("GREEN") ...and assigns it to the name, GREEN. Guess I'll leave it at that. Let's see how this is received. --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler

I made a mistake. This sentence: My idea is to optionally allow any callable object to write a
Should read: My idea is to optionally allow any callable object to write a __decoration_call__ method that gets called in lieu of the __call__ method when the callable object is employed using decorator syntax. When this happens, the decorated named is supplied- not counting self- as the *SECOND* argument (e.g., by_name), which contains the str value of the name the decorator was applied to. --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler On Wed, May 26, 2021 at 12:43 PM Ricky Teachey <ricky@teachey.org> wrote:

I'm still digesting this proposal (though I think I like it quite a bit), but there's one thing in particular that just really doesn't gel with me. Is there any particular reason why you've proposed assignment decorators as being on the same line as the assignment statement rather than on the preceding line(s)? I like this: @typing.TypeVar T = str, bytes about a million times better than: @typing.TypeVar T = str, bytes Because the latter feels (to me) too similar for comfort to this: int foo = 3 Which is in my mind not very pythonic. Also, my brain just has an easier time parsing the multiline version than the single-line one (though I concede that a combination of familiarity and syntax highlighting would solve that issue eventually). It also represents an asymmetry between the syntax of the proposed assignment decorators and the syntax for function and class decorators. And finally, it doesn't cleanly accommodate use-cases like those proposed by Stéfane in the previous thread: @acl(READ, WRITE) @constraint(10 < _ < 100) @not_null @indexed @depends_on(whatever) @inject first_name: str Whereas the multiline variant does. Another question I've got is whether you've got any solution in mind for more complex assignment cases? After thinking about the proposals in the Steven's thread (the googly-eye symbol), I think I'm a fan of providing the name as a *tuple of tuple of strings*, like so: @decorate foo = first, _, third, *rest = bar = [0, 1, 2, 3, 4] such that __decoration_call__ receives the following tuple for its second argument: (('foo',), ('first', '_', 'third', '*rest'), ('bar',) And it can then choose to do whatever it likes with all the available names without having to implement its own logic for parsing the names back out from a string. My only other question would be, what would you think of adding a third argument to pass the type hint along? Function decorators can always access the __annotations__ on the function object they act on, but currently your __decoration_call__ can't capture type information. With such an argument, this: @decorate foo: int = 3 Would be provided to 'decorate' as: def __decoration_call__(self, obj, names, annotation): print(obj) # 3 print(names) # (('foo',),) print(annotation) # int But yeah, overall I'm liking this proposal best out of the ones that have been discussed recently. The google-eyes symbol is a solid idea, and is sufficient for many basic use-cases, but it's not really as powerful as real decorators and cannot accommodate the more complex cases (like the example above with several chained decorators, some of which are actually decorator factories). On Wed, May 26, 2021 at 5:52 PM Ricky Teachey <ricky@teachey.org> wrote:

On Wed, May 26, 2021 at 2:22 PM Matt del Valle <matthewgdv@gmail.com> wrote:
No reason, very bikesheddable. I like this:
I do not as of yet. Still thinking about it.
I'd not be opposed. Another (fourth) argument could be the code object from the RHS (if inspect.getsource can't be shoehorned into getting it for us). --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler

Now this is a really interesting proposal. Something wasn't right in the other discussion, I didn't think making variable decorators inconsistent with the current class and function decorators by providing the variable name was particularly good. I've always felt like that something like __decoration_call__ was missing, so I'm really grateful that someone took the time to think about it. I'd say this looks pretty promising, and I can't see any specific downside compared to the alternative proposals.

On Wed, May 26, 2021 at 12:43:48PM -0400, Ricky Teachey wrote: [...]
Ricky, it's not clear to me whether you are proposing the above RESULT A and RESULT B as an *alternative* to the "variable decorator" proposal, or if you have just misunderstood it. The current variable decorator proposal on the table is for this: @decorator(spam) name # --> name = decorator("name", spam) rather than what you wrote: # name = decorator("spam")("name") So I can't tell whether the difference between your version and the OPs is a bug or a feature :-)
To be clear here, I think that your proposal is that this method is to be looked up on the *decorator*, not the thing being decorated. Is that correct? In other words: @decorator class X: ... # or a function, or something else it is *decorator*, not X, that is checked for a `__decoration_call__` method. Correct?
In current Python, the only objects which can be decorated with the @ syntax are callable functions and classes. So it is ambiguous to talk about "any callable object" without stating whether it is the decorator or the thing being decorated.
Roughly speaking, that would correspond to @decorator("spam this") def func(): ... If we have a bare decorator, we have this: @decorator def func(): ... # --> func = decorator.__call__(func)
Okay. Without reading the source code, does this code snippet use the old `__call__` protocol or the new `__decoration_call__` protocol? @flambé class Banana_Surprise: pass It seems to me that this proposal means that we can't even tell which of the two protocols (classic decoration, or new `__decoration_call__` style decoration) without digging into the implementation of the decorator. To be precise, the problem here as reader isn't so much the fact that I don't know whether the object is called using the `__call__` protocol or the new-style `__decorator_call__` protocol, but the fact that I can't tell whether the calls will involve the name being passed or not. This is because the name is being *implicitly* passed, in a way that is unclear whether or not it will be passed. I just don't know whether or not the decorator `flambé` receives the name or not.
What happens if the decorator factory has `__decoration_call__` and the object it returns only has `__call__`? I presume you get this: func = decorator.__decoration_call__("spam this", "func").__call__(func) And let's not forget the other two combinations: func = decorator.__decoration_call__("spam this", "func").__decoration_call__(func, "func") func = decorator.__call__("spam this").__call__(func) The last one is, of course, the current behaviour for a decorator factory. The bottom line here is that is you have plain, unadorned decorator: @decorator there are two possible behaviours and no obvious way to tell which one is used, short of digging into the implementation. But if you have a decorarator factory: @factory(*args, **kwargs) there are now four possible behaviours. And anyone brave enough to use a double-barrelled factory-factory @factory(*args, **kwargs)(*more_args) will be faced with eight possible combinations. And at this point, I'm afraid I have run out of steam to respond further to this proposal. Sorry. -- Steve

I'm not the OP, but the way I understand the proposal __decoration_call__ is only invoked when you actually *use an object to decorate something*. That means that a decorator factory will just invoke __call__ as normal, because it's nothing but a convenient way to generate a decorator. It is not itself a decorator, nor is it used to actually decorate anything. To illustrate this point we can separate it out across several lines: @factory("foo") def bar(): pass Can be rewritten as: decorator = factory("foo") @decorator def bar(): pass So __decorator_call__ will only be invoked on the object that gets returned from `factory("foo")`, not on `factory`. It seems to me that this proposal means that we can't even tell which of
The OP mentioned a default implementation for __decoration_call__ of: def __decoration_call__(self, func, by_name): if func is None: return self(by_name) return self(func) Such that you can assume that the decorator will *always* receive the name, but may choose to discard it and not make use of it if it doesn't implement the __decoration_call__ interface and instead opts to use default implementation which falls back on __call__. For decorated functions the name can always be pulled out of the function object as normal even when using __call__, but to make use of the name in a decorated assignment statement the decorator would have to override __decoration_call__. At this point I will say that I may be putting words into OPs mouth, and would be happy to be corrected if I've misunderstood. One final point I've just thought of is that Ricky suggested that when no value is assigned to a name that the object reference be `None`. But I don't think that works, because it becomes indistinguishable from when `None` is explicitly assigned. We would need some sentinel value instead of `None` to remove ambiguity in this situation: from somewhere import NOTSET @decorate foo: int def __decoration_call__(self, obj, names, annotation): print(obj is None) # False print(obj is NOTSET) # True @decorate foo: int = None def __decoration_call__(self, obj, names, annotation): print(obj is None) # True print(obj is NOTSET) # False On Thu, May 27, 2021 at 3:25 PM Steven D'Aprano <steve@pearwood.info> wrote:

On Thu, May 27, 2021 at 11:09 AM Matt del Valle <matthewgdv@gmail.com> wrote:
Correct.
Yes but I am on the fence as to whether this default implementation (I suppose it would live on the object class?) should be considered or not. It would certainly provide a lot of functionality "out-of-the-box". For decorated functions the name can always be pulled out of the function
Nah you got it.
Bikesheddable, but I don't know why having these two be equivalent: @decorator var @decorator var = None ..would be a problem. Having an implied default of None for var above makes sense to my brain. Do you have an example in mind where you think it would create a problem? --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler

I don't have anything immediately in mind, no, but I think that given the semantics of those two statements are very different, it is at least worthwhile to allow the decorator to know which of them is actually happening (is `None` being assigned to the name or is it maybe just a type hint for that name?). Keep in mind that if no assignment is happening then there is no need to return anything from the decorator, since it will just be lost anyway. And I'm pretty much 100% positive that even if I can't think of a use-case off the top of my head, there will eventually be a library author (if this proposal is accepted) who will have some cool idea that will require distinguishing these two scenarios. Like for example if this were used for a CLI-building library (think something like Typer) how the assignment to `None` could signify that it is an option with a default value of `None`, whereas the bare name would signify that it is a mandatory argument. Oh, and I think I've just discovered another thing that I'm not 100% sure I like. Even putting aside that I'm not a fan of decorators on the same line as the statement they are decorating (as I mentioned in an earlier response), you've got examples of variable decorators where no assignment is happening such as: @decorator var To me this breaks the symmetry between function decorators, which always decorate a function definition (an implicit form of assignment), and the proposed variable decorators. They are also confusing in the sense that the decorator is de-facto turning an otherwise invalid python statement legal. If you remove the decorator from the above example you will presumably get a `NameError`. I imagine this would then have to be special-cased somehow in the language spec so that an undefined name is not evaluated, but only when preceded by a decorator? I don't know, it seems messy to me. Also, I just can't quite see the value in them if I'm honest, whereas the version that is applied to an assignment statement: @decorator var: bool = True And even a bare type-hint version: @decorator var: bool seem to me to be far more self-evidently useful. On Thu, May 27, 2021 at 5:45 PM Ricky Teachey <ricky@teachey.org> wrote:

On Thu, May 27, 2021 at 1:40 PM Matt del Valle <matthewgdv@gmail.com> wrote:
...
Oh, and I think I've just discovered another thing that I'm not 100% sure I
Ok, agreed on all points. I think an eventual full-fledged proposal could easily put naked decorations like: @decorator var ...to the wayside, to be added later if people have a really good reason for it. --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler

On Thu, May 27, 2021 at 10:40 AM Matt del Valle <matthewgdv@gmail.com> wrote:
I agree that if we allow both then they should be distinguished somehow. They are different not just in value but in existence since on the next line down "var" will either be just fine or raise a NameError. If the decorator is expecting for some reason to reach into its parent scope and modify this variable, those are two very different situations.
I am confused why you are okay with @decorator var: bool but not @decorator var Yes, a bare name is currently an error while just a name and a type hint is valid, but the latter doesn't bind anything to the name, and using that identifier is still a NameError. So a decorator presumably can't return a value for either (or it could, but it would always be dropped). What could it do with a name and a type hint that is so much better than just a name? I am still very confused as to the scope of this counter proposal re variable decorating. I have only seen two such examples here @decorator variable # variable = decorator.__decoration_call__(None, "variable") @decorator variable = "spam" # variable = decorator.__decoration_call__(variable, "variable") But what is actually valid to follow a decorator in this proposal? Any simple expression, any expression? Is it limited to assignment espressions? Here are some interesting uses that were brought up in the other thread and I would like to know how they would work. @decorator spam = eggs = cheese = "tomatoes" @decorator spam, eggs, cheese = "tomatoes" @decorator spam, eggs = cheese = "tomatoes" @decorator spam = (eggs := "cheese) @decorator locals()[find_it() or "default"] = spam() Regards, ~Jeremiah

On Thu, May 27, 2021 at 10:25 AM Steven D'Aprano <steve@pearwood.info> wrote:
No, I understood the OP's proposal perfectly. I was agreeing with you implicitly when you previously said the inconsistency between the OP's proposal and current decorator is a problem: <STEVE WROTE>: On the other hand, it would be terribly confusing if the same syntax:
And furthermore, it is even a bigger problem than you explicitly made it out to be because of passing and object vs. passing a string representing the name of an object. So I was not illustrating the OP's proposal, but showing (but not proposing) a modified version of it that acts more like decorators today, so that we would have this: # NOT THE OP's PROPOSAL -- more consistent with decorators today, but not as quickly useful on its own @decorator("spam this") var # decorator("spam this")("var") ..rather than this: # OP's PROPOSAL @decorator("spam this") var # decorator("spam this", "var")
Yes, I am not proposing this behavior or saying it was the OP's, just illustrating to agree with you that it is so different from current decorator behavior, I don't think it should be considered (even though I sort of like how it looks).
Yes, it is looked up on the decorator, i.e., the result of the expression immediately to the right of the @ symbol. So it is only used when using *decorator syntax* (i.e., the @ symbol).
Yes.
I am sorry, I thought I made this clear early on in the proposal. yes, the decorator is the thing. More precisely, the result of the expression to the right of the @ symbol. By "any callable", I had in mind how any callable can be used as a decorator (so long as it isn't expecting more than 1 positional argument and no required kwd arguments). But really, the result any expression can be a decorator now (as of 3.10 I believe?), thought you might get an error: @1/2 def func(): ... # TypeError: 'float' object is not callable
Yes, I had written exactly that above the quoted part: So, for any existing callable with the name decorator, this:
Continuing on.
Invoking @flambé causes: flambé.__decoration_call__(Banana_Surprise, "Banana_Surprise") ...to be used, if flambé has that method.
HMMM.... to be totally honest I had not considered the problem this idea poses for introspection and documentation. But after giving it some thought, I don't see it as a significant problem. Yes, you would have to read the source code of flambé, or consult help/introspection tools that know about the new protocol, to know whether a __decoration_call__ method will be used. And introspection/help tools would need to present both the normal call signature: flambé() <---> flambé.__call__(self, spam, /, eggs, *args, cheese, **kwargs) ....and the decoration call signature, which is intended to be FIXED at two position args: @flambé <---> flambé.__decoration_call__(self, func, by_name) Of course, anyone could make the __decoration_call__ signature anything they wanted: @flambé <---> flambé.__decoration_call__(self, func, /, by_name, *chaos, pain="why?", **suffering) I do see that as a potential problem now; but I think it's a minor problem. Help and tutorial guides/files would have to teach people that there is potentially a different calling signature when decorator syntax is used, just as they teach that other syntactical symbols call different dunder methods when they are used: a += 1 a + 1 But I think the flexibility we will get could be worth this didactic overhead.
If you wrote the decorator, you know. If you didn't, why does it matter if it is being passed? Why do you need to know that? Some descriptor have __set_name__ methods, and others don't. We use them very happily in both cases. The callable function you are using as a decorator has a job; presumably you know what the job is or you'd not be using the callable. Do you really have to concern yourself with HOW it does it, unless you are the maintainer, or tracking down a bug inside the implementation of the decorator? At which point you'll have to read the source code anyway.
No. I am NOT-- repeat, NOT-- proposing a change to the behavior of the call operator, (). The call operator, (), ALWAYS does this: decorator_factory() <--> decorator_factory.__call__() This is true regardless of whether that call is prepended by the @ symbole. Clear? Under NO circumstances does: decorator_factory("spam this") ...result in this: decorator_factory.__decoration_call__("spam this") No. ONLY prepending the @ symbol at the beginning of a line invokes __decoration_call__, and it invokes it on the result of the expression immediately to the right, just as it does today: @decorator_factory() <--> decorator_factory().__decoration_call__ @( decorator := decorator_factory() ) <--> ( decorator := decorator_factory() ) .__decoration_call__ @tricky_decorator + 1 / 2 "lala" ^ None <--> ( tricky_decorator + 1 / 2 "lala" ^ None ).__decoration_call__ The object that is returned by the factory object is the decorator. NOT the decorator factory. These code blocks result in identical behavior for func: decorator = decorator_factory("spam this") @decorator def func(): ... @(decorator := decorator_factory("spam this")) def func(): ... @decorator_factory("spam this") def func(): ... The object returned by the expression to the right of the @ symbol is not in any way affected by the proposal. Everything happens as it does today. Zero changes.
I said it above already but I don't see why this is such a problem. Presumably you are using this callable as a decorator because you are confident to some degree it will do the job it is supposed to do. If it needs the by_name argument to do that job, it will make use of the __decoration_call__ dunder. If it doesn't need it, it won't.
No, as I explained above, that isn't the case. The decorator object returned by the factory call has two possible behaviors, yes. And yes, understand which happens, you have to look at how it is implemented.
No; it remains two: factory.__call__(*args, **kwargs) .__call__ (*more_args) .__decoration_call__ OR, the default: factory.__call__(*args, **kwargs) .__call__ (*more_args) .__call__
I don't blame you. The version of my proposal you had in mind was terrible; sorry I did not explain it so well the first time. I hope the version I actually have in mind will be better received. --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler

On Thu, May 27, 2021 at 9:34 AM Ricky Teachey <ricky@teachey.org> wrote:
I got the sense that people both liked reading my examples of same-line decorators and pushed back against not making them appear just like function decorators. One way to have my cake and you eat it too would be to relax the current decorator grammar to not require a NEWLINE. AFAICT there would be no ambiguity since after a decorator there must either be another "@" or a "def". Then both function and assignment decorating can be both. These would all be possible and not change the status quo. @cache def factorial(n): pass # This is your example @namedtuple Point = "x y z" # OP was @namedtuple("x y z") Point @not_null @indexed first_name: str
Actually the original proposal was @decorator("spam this") var # var = decorator("var", "spam this") The implied assignment that is not written was a big part of the proposal, as most of my examples were defining the name for the first time, and also the original proposal disallowed assignment (or any statement) after the decorator; it is only valid to have an identifier after. Also the "var" coming first is a subtle but important distinction. By providing the name as the first argument, all of my examples of callables currently in the standard library will work as you say out of the box. If it were to be passed in last, this new syntax would not be usable by any standard library callable (or even third party? Does anyone create factory functions that need the name and take it last?) and lots of new functions would have to be added. Regards, ~Jeremiah

On Thu, May 27, 2021 at 2:00 PM micro codery <ucodery@gmail.com> wrote:
...
By providing the name as the first argument, all
Yes that's true. I originally wrote the proposal with the arguments switched around. But I it seemed like it could be important to have the decorated_object argument appear first in the signatures for both __call__ and __decoration_call__, or it could cause confusion... However, I am open to the idea I was right the first time. If so, the default implementation at the top of the object food chain would be something like (using Matt's SENTINEL idea): def __decoration_call__(self, by_name, obj=SENTINEL): if func is SENTINEL: return self(by_name) return self(obj) On the one hand, this might be too expensive; an identity check and potentially two function calls occur for all @ decorations. And I don't see how to optimize this away with opcodes... On the other hand, how often are people invoking millions of @ decorations in a loop...? --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler

On Thu, May 27, 2021 at 12:33:20PM -0400, Ricky Teachey wrote:
Right: you can't tell from the code you are looking at whether flambé is being called as a new or old style decorator. You have to dig into the definition of flambé to work out the meaning of the code. On the one hand we have the current decorator protocol, where the object flambé is called with a single argument, the class Banana_Surprise. I can only think of three cases where something similar occurs, but the differences are critical. - Binary operators can call either or both of two reflected dunders, such as `__add__` and `__radd__`. - Iteration normally calls `__next__`, but it can fall back on the old sequence protocol that relies on `__getitem__`. - Augmented assignment calls the augmented assignment method, such as `__iadd__`, but may fall back on the regular `+` operator. In all of these cases, the intended semantics are the same and the arguments passed are (almost) the same. E.g. normally we would use the same signatures for each of the operators: def __add__(self, other): ... def __radd__(self, other): ... def __iadd__(self, other): ... and if fact for many purposes, add and radd will be the same function object. The old sequence protocol is a minor exception, because `__getitem__` requires an argument (it is called with 0, 1, 2, 3, ... until IndexError occurs). But that's almost an implementation detail. The fact that `__next__` takes no explicit enumeration, but `__getitem__` does, can be ignored when we think about the high-level concept of iteration. Conceptionally, at a high-level, the sequence and iterator protocols operate the same way: call the dunder repeatedly until an exception occurs and the details of which dunder and which exception don't matter. Iteration is iteration. But here, the details *do* matter. Old classic decorators and this proposed new decorator syntax do not share a high-level concept. There is a certain relationship in the sense that both the new and old protocols involve calling a function, but the details are very different: - old decorator protocol is a normal function call with a single argument; - new protocol uses a different method, with two arguments, one of which is implicit. Old decorator protocol is about *decorating* a function or class object with new functionality (wrapping it in a wrapper function, say, or modifying it in place). New decorator protocol and variable decoration doesn't decorate anything: there's nothing to decorate when used with a bare name: @decorate var which presumably evaluates to: var = decorate.__decorate_call__('var') So we're not decorating an object. There's no object to decorate! With a binding: @decorate var = expression var = decorate.__decorate_call__('var', expression) Again, conceptually we're not decorating anything. We're just reusing the `@` syntax for something that is not conceptually related to old style decorators. This is just a syntax for *implicitly* getting access to the left hand side assignment target name as a string. [...]
Because I don't know the meaning of the code. Am I decorating an object or doing something with the target name? I can't interpret the code until I know what protocol was supported by the decorator object. Let me give you an analogy: the `with` statement has a context manager protocol that uses an `__enter__` and `__exit__` pair of methods. With statements are awesome! They're so awesome that we should use the exact same "with" syntax for Pascal-style implicit attribute access, using a different protocol. https://docs.python.org/3/faq/design.html#why-doesn-t-python-have-a-with-sta... with myobject: do_something() with myotherobj: do_something() There is no visible difference between the two pieces of code, but one is an old style enter/exit context manager, the other is our hypothetical new style with syntax that does something completely different and calls different dunder methods. One of them is calling the global do_something and one is calling the object's do_something method. Which is which? Code that is different should look different. -- Steve

On Tue, Jun 1, 2021 at 9:57 PM Steven D'Aprano <steve@pearwood.info> wrote:
My understanding is that it would attempt to invoke __detonation_call__ (err, I mean, __decoration_call__, but just think how awesome the other would be) first, and if it can't find it, it falls back on regular __call__. That would parallel how most other things are done - repr falling back to str, iadd falling back to add, etc. That said, I still don't like the idea of decorating a bare name. A magic "assignment target" token that translates into the string form of the thing being assigned to is far more useful and far less magical. ChrisA

On Tue, Jun 01, 2021 at 10:17:32PM +1000, Chris Angelico wrote:
Sure, but the analogy is more like trying repr first to produce a string, and if that doesn't work, trying iter to produce a list. Function/class decoration shares little with "variable decoration" except the name and (proposed) use of the @ syntax. It seems to me that they are only the most loosely related concepts. Ricky's proposal to use a new dunder magnifies the differences and obfuscates the similarities.
Obviously I agree, given that I proposed a special @@ symbol to do that however Stéfane's real-world example of chaining variable decorators has forced me to take that idea seriously. @acl(READ, WRITE) @constraint(10 < _ < 100) @not_null @indexed @depends_on(whatever) @inject @... first_name: str https://mail.python.org/archives/list/python-ideas@python.org/message/DCKFMW... -- Steve

Another problem I have with this proposal is the new dunder method. Why do we need a new dunder? We're probably never going to "decorate" a variable and function with the same callable, the two use-cases are very different. But even if we do, we can easily distinguish the two cases. Let's dump the `__decorate_call__` dunder and just use normal call syntax. That will be a HUGE win for useability: any function, class or callable object can be used as the decorator, it doesn't have to be a custom object with a custom class that defines a special dunder. We can distinguish the two contexts by using different signatures. The signature used depends entirely on the call site, not the decorator, so it is easy for the interpreter to deal with. If the decorator is called on a function or class statement, a single argument is always passed, no exceptions: # always calls decorate with one argument @decorate def func(): # or class pass # --> func = decorate(func) If called on a variable, the number of arguments depends on whether it is a bare name, or a value and annotation are provided. There are exactly four cases: # bare name @decorate var # --> var = decorate('var') # name with annotation @decorate var: annot # --> var = decorate('var', annotation=annot) # name bound to value @decorate var = x # --> var = decorate('var', value=x) # name with annotation and value @decorate var: annot = x # --> var = decorate('var', annotation=annot, value=x) Keyword arguments are used because one or both of the value and the annotation may be completely missing. The decorator can either provide default values or collect keyword arguments with `**kwargs`. The only slightly awkward case is the bare variable case. Most of the time there will be no overlap between the function/class decorators and the bare variable decorator, but in the rare case that we need to use a single function in both cases, we can easily distinguish the two cases: def mydecorator(arg, **kwargs): if isinstance(arg, str): # must be decorating a variable ... else: # decorating a function or class assert kwarg == {} So it is easy to handle both uses in a single function, but I emphasise that this would be rare. Normally a single decorator would be used in the function/class case, or the variable case, but not both. -- Steve

On Tue, Jun 1, 2021 at 10:16 PM Steven D'Aprano <steve@pearwood.info> wrote:
I can't imagine any situation where you would *want* this, but it is actually possible for a decorator to be given a string:
This probably falls under "you shot yourself in the foot, so now you have a foot with a hole in it". ChrisA

On Tue, 1 Jun 2021 at 13:16, Steven D'Aprano <steve@pearwood.info> wrote:
I've yet to be convinced that variable annotations are sufficiently useful to be worth all of this complexity (and by "this" I mean any of the proposals being made I'm not singling out Steven's suggestion here). But if we do need this, I quite like the idea of making the distinction based on signature.
You don't need to do this. Just add another keyword argument "name": # bare name @decorate var # --> var = decorate(name='var') # name with annotation @decorate var: annot # --> var = decorate(name='var', annotation=annot) # name bound to value @decorate var = x # --> var = decorate(name='var', value=x) # name with annotation and value @decorate var: annot = x # --> var = decorate(name='var', annotation=annot, value=x) The single positional argument is reserved for function/class annotations, and will always be None for variable annotations. Paul

On 2021-05-26 09:43, Ricky Teachey wrote:
This seems contradictory to me. It looks like you're saying, "We shouldn't use decorator syntax to represent two different things (object vs name), but instead decorator syntax should give us access to two different things (object vs name)." I realize based on your proposal there is a distinction here but I think it's quite a narrow one and doesn't resolve the basic problem, which is that currently decorators operate on objects and these new proposals are about making them operate on names. I think there may be value in having some feature that lets us get access to the name side of an assignment. But I wouldn't call such a thing a "decorator", nor would I want to use the same @ syntax that is used for decorators. To me that would be confusing, because the behavior is totally different. Even with your __decorator_call__ proposal, there's still a jarring shift from, in some cases, using just the object, and in other cases stuffing a new parameter (the name) into the parameter list. That seems awkward to me. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown

Whoops, replying all this time. On Thu, May 27, 2021 at 2:32 PM micro codery <ucodery@gmail.com> wrote:
At this point, I have in mind any expression that appears to the right, which I believe is what is allowed today: @1/2 "lala" and money def func(): ...
I would also like to know how all of these work :D I am not sure about most of them but open to suggestions. The only one that I feel confident about is: @decorator spam = (eggs := "cheese) ...which, I think, should be: decorator.__decoration_call__(spam, "spam") Unfortunately for the proposal most people don't seem too thrilled with it. So I don't plan to spend a lot of time thinking through these examples and suggesting behavior. Anyone is welcome to do that though, this isn't MINE in the sense I am jealously guarding ownership of the details. :) On Thu, May 27, 2021 at 3:03 PM Brendan Barnwell <brenbarn@brenbarn.net> wrote:
Yes, and for this reason I really liked Steve's googly eyes proposal in the other thread. But I wonder if there value in specifically giving decorators access to the name side? It seems to me that it would open up a lot of possibilities, just as when descriptors learned their names. class Desc: def __set_name__(self, owner, name): self.name = name def __get__(self, instance, owner): if instance is None: pass print(f"I am { owner.__name__}.{self.name}") class C: v = Desc()
C().v I am C.v
We could make the __decoration_call__ method even more powerful. We could give it access not just to the name side, but to the type info, and even the code object/expression side (i.e., RHS). @decorator x: Fraction = 1/2 # decorator.__decoration_call__(x, "x", "1/2", Fraction) So, for example, a Math library can create the feature: @Math x: Fraction = 1/2 And x is: Fraction(1, 2) Yes, you can do: x = Fraction("1/2") ...today. I get that. But it's not as if this Math example is the ONLY thing it allows you to do. You also can easily avoid repeating things and making mistakes: @namedtuple_factory Point = "x y" I'm sure there are many other things I haven't thought of. --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler On Thu, May 27, 2021 at 3:03 PM Brendan Barnwell <brenbarn@brenbarn.net> wrote:

To reiterate my point from earlier in the thread, I am quite firmly opposed to having the decorator on the same line as the statement being decorated. I'll just copy-paste my reasoning for it: I like this:
And regarding your proposal to relax the newline requirement on function/class decorators: I got the sense that people both liked reading my examples of same-line
This would admittedly resolve the asymmetry between the proposed single-line variable decorator syntax and current decorators, but personally I just *really* don't like it. The only benefit is saving a line, but it comes at (in my opinion, your mileage may vary) a huge cost to legibility, and it goes against the zen: There should be one-- and preferably only one --obvious way to do it. And it's just something that has (almost) no precedent in python (aside from `async def`). It has the feel of something like: public static final native abstract void SomeMethod() {...} I'm imagining a future of reading python code like: @too @many("!") @decorators @on("a", "single", "line") def foo(...): And it makes me unhappy :( So going back to your original point of why I'm okay with decorating a bare type-hint, like: @decorate foo: int But not @decorate foo The reason is simply that if you take the decorator away the first one is legal, and the second one will raise `NameError`. I'm happy to decorate a statement that is valid on its own, but I'm against the idea of special-casing decorator syntax so that it can decorate otherwise-invalid statements. And I do think that there are legitimate uses for decorating a bare type-hint, since it does actually contain useful information the decorator might want to capture (for example, in one of its own instance attributes if the decorator is an object rather than a function, which it would have to be in order to implement __decoration_call__). You're right in that it wouldn't be able to return anything since no assignment is taking place, but there are still potential use-cases for it. I'll concede that there are also use-cases for decorating a completely naked name, but none of the ones I've seen so far seem to me compelling enough to break the rules like this. On Thu, May 27, 2021 at 8:43 PM Ricky Teachey <ricky@teachey.org> wrote:

On Thu, May 27, 2021 at 12:39 PM Ricky Teachey <ricky@teachey.org> wrote:
I don't think I was clear enough with my original question. I agree that any valid expression can follow the @, just like function decorators; I said as much in my original proposal. My question is what can follow after the line "@expression"? Could it be a with block; a lambda; a match statement? Probably just an assignment statement, but even just that covers a lot including multi-assignment or including a yield expression. But I think I know your answer to this from your reply to my later examples ;-) On Thu, May 27, 2021 at 2:19 PM Matt del Valle <matthewgdv@gmail.com> wrote:
Yes, there seems to have been a lot of rejection of the single-line decorator. I liked it for variable decorating because I didn't think developers would use it if it meant adding an extra line to every variable declaration that wanted to take advantage of the new syntax. Also, limiting its use to only identifiers kept the lines short. But I agree that it could lead some very unpythonic code, especially if multiple decorators on the same line were allowed. I just thought it looked better, but would you and others here like it better if the NEWLINE requirement was kept for all decorators? There is nothing in the original proposal that requires a single line. I also don't know what should happen for complicated assignments, and I think this has been the death of such variable decorator discussions in the past, so I would still push for only bare identifiers, with or without a type hint (but maybe it will be better received by more if the type hint is required?). I still think such a proposal is strong enough on its own to be of value to the language. So now the syntax would be: @decorator variable: Type @decorator("spam", eggs) variable: Type become variable = decorator("variable") variable = decorator("spam", eggs)("variable") I'm not sure that the type hint shouldn't be passed as an additional parameter, especially if one is always required. Yes it is still different from a function decorator in that you are not getting the variable object (which may or may not exist) but instead its name as a string. However, decorators that expect to be applied to variables should already be drastically different. Even if they did get the variable object, there would be not necessarily be a __code__ or __name__ or __dict__ attribute. They won't be callable (generally) where most current decorators attempt to call func, wrapped in an internal closure. In this case Stéfane's long example would still be valid, and some of the one-liners would instead look like this: class Colors(Enum): @str RED: str @str GREEN: str @str BLUE: str @namedtuple("x y z") Point: NamedTuple @os.getenv PATH: Optional[str] Regards, ~Jeremiah

On Fri, May 28, 2021 at 5:07 PM Rob Cliffe Co<rob.cliffe@btinternet.com> wrote:
Fair enough! If this became accepted syntax I would use it without type hints. Even for those of us that do use type hints in places, it shouldn’t generally be necessary because the decorator will have a return type annotation. I think the original argument was that currently bare identifiers are not allowed unless they have annotation. But this is introducing a new multiline syntax, and it makes no more sense to take away the second line and expect a naked decorator to be valid than it does remove the decorator and expect the naked identifier to be valid.

Ah, I think you might be missing the context of the original proposal? I do mean bare unbound identifiers - at lease as they occur in this new syntax. # currently works spam = “eggs” spam: eggs # currently a NameError spam # proposed to work, currently illegal @spam eggs @spam(“eggs”) cheese @spam eggs: str But none of this would change the first three examples.

I think there's a fundamental difference between your original proposal and the OPs proposal in this thread, which is that you seem to envision variable decorators as granting access to just a name (and potentially a type-hint?), whereas the OP wants the decorators extended to assignment in general. Basically, your proposal is a subset of the OPs proposal. Any implementation of the OPs decorators would also allow us to do all the things you want to be able to do with variable decorators (as a happy side-effect), but would also offer a LOT more functionality. My view is that variable decorators should be legal for: 1) assignment statements (so that the decorator can modify or replace the object that is getting assigned) 2) bare type-hints, since these are arguably *a sort of* assignment operation that still carries useful info a decorator might want to capture (you're not assigning the name and its value to globals(), but rather the name and its type to globals()['__annotations__']) If there was huge pushback against point 2 I'd be happy to see it deferred and revisit it at a later point, but as far as I'm concerned point 1 is what actually interests me about this proposal. Your point about complicated assignment being a problem seems solvable using the following rules: 1) Variable decorators only capture the name (or names) on the LHS of a true assignment statement (or bare type-hint). Expressions that happen to bind names (such as the walrus operator, for loop, etc.) don't have their names available for use in the decorator. 2) The proposed __decoration_call__ is given the value, name, and type-hint from the decorated statement (and potentially the code object from the RHS, as OP suggested). We could provide the names as a series of useful pre-parsed parsed AST-like objects, like: @decorate some_obj.foo = first, _, third, *some_obj.rest = some_dict[(lost_name := ('ba' + 'r'))] = baz = (another_lost_name := [0, 1, 2, 3, 4]) such that __decoration_call__ receives: def __decoration_call__(self, obj, names, annotation): print(obj) # [0, 1, 2, 3, 4] print(names) # ParsedNames(names=[ # ObjectAssignment(obj={reference_to_some_obj}, name='foo'), # DestructuringAssignment(names=[ # BasicAssignment(name='first'), # BasicAssignment(name='_'), # BasicAssignment(name='third'), # ObjectAssignment(obj={reference_to_some_obj}, name='*rest') # ]), # ItemAssignment(obj={ref_to_some_dict}, name='bar'), # BasicAssignment(name='baz') # ]) print(annotation) # NOTSET Notice that as per point 1 above, `lost_name` and `another_lost_name` do not get captured, because they are not part of the assignment statement. They're only sub-expressions. Also, notice that expressions like 'ba' + 'r' are evaluated before being passed to the decorator, so that it actually receives 'bar' for the __setitem__ assignment name. 3) Augmented assignment gets expanded out to its true meaning before being passed to the decorator, so for example: foo = 3 @decorate foo += 2 behaves exactly the same as: foo = 3 @decorate foo = foo + 2 If there are still any issues given these rules that I can't think of I'd be happy for people to raise them. There are so many edge-cases in python assignment that I could very easily be forgetting about something. But I think this covers 99% of cases. class Colors(Enum):
With decorated assignments these examples could instead look something like: class Color(Enum): @string_values RED, GREEN, BLUE = auto() Here the decorator would return a tuple like: ('RED', 'GREEN', 'BLUE'), which would be assigned instead of whatever is actually on the RHS And for the namedtuple: @namedtuple Point = 'x', 'y', 'z' The third one would look just like you suggested. On Sat, May 29, 2021 at 3:28 AM micro codery <ucodery@gmail.com> wrote:

On Sat, May 29, 2021 at 12:46:11PM +0100, Matt del Valle wrote:
Can you explain what benefit a variable decorator in this case would have over a function call? @func var = expression is, I expect, the same as: var = func(expression) or with the assignment target magic: var = func('var', expression)
Once we move out of bare type hints into calling a function, it's not a bare type hint any more. @func var: int if it calls func, is not the same as var: int which has no runtime effect. A note on augmented assignment:
3) Augmented assignment gets expanded out to its true meaning before being passed to the decorator, so for example:
The true meaning of augmented assignment is not to expand the line of code out to assignment with the operator, but a method call: target += expression # not this target = target + expression # but this instead: target = type(target).__iadd__(target, expression) Only if `__iadd__` doesn't exist does the interpreter fall back on the plus operator. This is why classes can implement augmented assignment as inplace operators: L = [1, 2, 3] L += [4] # modifies L in place L = L + [4] # creates a new list and assigns to L This is a significant difference as soon as you have multiple references to the original list. -- Steve

I made a mistake. This sentence: My idea is to optionally allow any callable object to write a
Should read: My idea is to optionally allow any callable object to write a __decoration_call__ method that gets called in lieu of the __call__ method when the callable object is employed using decorator syntax. When this happens, the decorated named is supplied- not counting self- as the *SECOND* argument (e.g., by_name), which contains the str value of the name the decorator was applied to. --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler On Wed, May 26, 2021 at 12:43 PM Ricky Teachey <ricky@teachey.org> wrote:

I'm still digesting this proposal (though I think I like it quite a bit), but there's one thing in particular that just really doesn't gel with me. Is there any particular reason why you've proposed assignment decorators as being on the same line as the assignment statement rather than on the preceding line(s)? I like this: @typing.TypeVar T = str, bytes about a million times better than: @typing.TypeVar T = str, bytes Because the latter feels (to me) too similar for comfort to this: int foo = 3 Which is in my mind not very pythonic. Also, my brain just has an easier time parsing the multiline version than the single-line one (though I concede that a combination of familiarity and syntax highlighting would solve that issue eventually). It also represents an asymmetry between the syntax of the proposed assignment decorators and the syntax for function and class decorators. And finally, it doesn't cleanly accommodate use-cases like those proposed by Stéfane in the previous thread: @acl(READ, WRITE) @constraint(10 < _ < 100) @not_null @indexed @depends_on(whatever) @inject first_name: str Whereas the multiline variant does. Another question I've got is whether you've got any solution in mind for more complex assignment cases? After thinking about the proposals in the Steven's thread (the googly-eye symbol), I think I'm a fan of providing the name as a *tuple of tuple of strings*, like so: @decorate foo = first, _, third, *rest = bar = [0, 1, 2, 3, 4] such that __decoration_call__ receives the following tuple for its second argument: (('foo',), ('first', '_', 'third', '*rest'), ('bar',) And it can then choose to do whatever it likes with all the available names without having to implement its own logic for parsing the names back out from a string. My only other question would be, what would you think of adding a third argument to pass the type hint along? Function decorators can always access the __annotations__ on the function object they act on, but currently your __decoration_call__ can't capture type information. With such an argument, this: @decorate foo: int = 3 Would be provided to 'decorate' as: def __decoration_call__(self, obj, names, annotation): print(obj) # 3 print(names) # (('foo',),) print(annotation) # int But yeah, overall I'm liking this proposal best out of the ones that have been discussed recently. The google-eyes symbol is a solid idea, and is sufficient for many basic use-cases, but it's not really as powerful as real decorators and cannot accommodate the more complex cases (like the example above with several chained decorators, some of which are actually decorator factories). On Wed, May 26, 2021 at 5:52 PM Ricky Teachey <ricky@teachey.org> wrote:

On Wed, May 26, 2021 at 2:22 PM Matt del Valle <matthewgdv@gmail.com> wrote:
No reason, very bikesheddable. I like this:
I do not as of yet. Still thinking about it.
I'd not be opposed. Another (fourth) argument could be the code object from the RHS (if inspect.getsource can't be shoehorned into getting it for us). --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler

Now this is a really interesting proposal. Something wasn't right in the other discussion, I didn't think making variable decorators inconsistent with the current class and function decorators by providing the variable name was particularly good. I've always felt like that something like __decoration_call__ was missing, so I'm really grateful that someone took the time to think about it. I'd say this looks pretty promising, and I can't see any specific downside compared to the alternative proposals.

On Wed, May 26, 2021 at 12:43:48PM -0400, Ricky Teachey wrote: [...]
Ricky, it's not clear to me whether you are proposing the above RESULT A and RESULT B as an *alternative* to the "variable decorator" proposal, or if you have just misunderstood it. The current variable decorator proposal on the table is for this: @decorator(spam) name # --> name = decorator("name", spam) rather than what you wrote: # name = decorator("spam")("name") So I can't tell whether the difference between your version and the OPs is a bug or a feature :-)
To be clear here, I think that your proposal is that this method is to be looked up on the *decorator*, not the thing being decorated. Is that correct? In other words: @decorator class X: ... # or a function, or something else it is *decorator*, not X, that is checked for a `__decoration_call__` method. Correct?
In current Python, the only objects which can be decorated with the @ syntax are callable functions and classes. So it is ambiguous to talk about "any callable object" without stating whether it is the decorator or the thing being decorated.
Roughly speaking, that would correspond to @decorator("spam this") def func(): ... If we have a bare decorator, we have this: @decorator def func(): ... # --> func = decorator.__call__(func)
Okay. Without reading the source code, does this code snippet use the old `__call__` protocol or the new `__decoration_call__` protocol? @flambé class Banana_Surprise: pass It seems to me that this proposal means that we can't even tell which of the two protocols (classic decoration, or new `__decoration_call__` style decoration) without digging into the implementation of the decorator. To be precise, the problem here as reader isn't so much the fact that I don't know whether the object is called using the `__call__` protocol or the new-style `__decorator_call__` protocol, but the fact that I can't tell whether the calls will involve the name being passed or not. This is because the name is being *implicitly* passed, in a way that is unclear whether or not it will be passed. I just don't know whether or not the decorator `flambé` receives the name or not.
What happens if the decorator factory has `__decoration_call__` and the object it returns only has `__call__`? I presume you get this: func = decorator.__decoration_call__("spam this", "func").__call__(func) And let's not forget the other two combinations: func = decorator.__decoration_call__("spam this", "func").__decoration_call__(func, "func") func = decorator.__call__("spam this").__call__(func) The last one is, of course, the current behaviour for a decorator factory. The bottom line here is that is you have plain, unadorned decorator: @decorator there are two possible behaviours and no obvious way to tell which one is used, short of digging into the implementation. But if you have a decorarator factory: @factory(*args, **kwargs) there are now four possible behaviours. And anyone brave enough to use a double-barrelled factory-factory @factory(*args, **kwargs)(*more_args) will be faced with eight possible combinations. And at this point, I'm afraid I have run out of steam to respond further to this proposal. Sorry. -- Steve

I'm not the OP, but the way I understand the proposal __decoration_call__ is only invoked when you actually *use an object to decorate something*. That means that a decorator factory will just invoke __call__ as normal, because it's nothing but a convenient way to generate a decorator. It is not itself a decorator, nor is it used to actually decorate anything. To illustrate this point we can separate it out across several lines: @factory("foo") def bar(): pass Can be rewritten as: decorator = factory("foo") @decorator def bar(): pass So __decorator_call__ will only be invoked on the object that gets returned from `factory("foo")`, not on `factory`. It seems to me that this proposal means that we can't even tell which of
The OP mentioned a default implementation for __decoration_call__ of: def __decoration_call__(self, func, by_name): if func is None: return self(by_name) return self(func) Such that you can assume that the decorator will *always* receive the name, but may choose to discard it and not make use of it if it doesn't implement the __decoration_call__ interface and instead opts to use default implementation which falls back on __call__. For decorated functions the name can always be pulled out of the function object as normal even when using __call__, but to make use of the name in a decorated assignment statement the decorator would have to override __decoration_call__. At this point I will say that I may be putting words into OPs mouth, and would be happy to be corrected if I've misunderstood. One final point I've just thought of is that Ricky suggested that when no value is assigned to a name that the object reference be `None`. But I don't think that works, because it becomes indistinguishable from when `None` is explicitly assigned. We would need some sentinel value instead of `None` to remove ambiguity in this situation: from somewhere import NOTSET @decorate foo: int def __decoration_call__(self, obj, names, annotation): print(obj is None) # False print(obj is NOTSET) # True @decorate foo: int = None def __decoration_call__(self, obj, names, annotation): print(obj is None) # True print(obj is NOTSET) # False On Thu, May 27, 2021 at 3:25 PM Steven D'Aprano <steve@pearwood.info> wrote:

On Thu, May 27, 2021 at 11:09 AM Matt del Valle <matthewgdv@gmail.com> wrote:
Correct.
Yes but I am on the fence as to whether this default implementation (I suppose it would live on the object class?) should be considered or not. It would certainly provide a lot of functionality "out-of-the-box". For decorated functions the name can always be pulled out of the function
Nah you got it.
Bikesheddable, but I don't know why having these two be equivalent: @decorator var @decorator var = None ..would be a problem. Having an implied default of None for var above makes sense to my brain. Do you have an example in mind where you think it would create a problem? --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler

I don't have anything immediately in mind, no, but I think that given the semantics of those two statements are very different, it is at least worthwhile to allow the decorator to know which of them is actually happening (is `None` being assigned to the name or is it maybe just a type hint for that name?). Keep in mind that if no assignment is happening then there is no need to return anything from the decorator, since it will just be lost anyway. And I'm pretty much 100% positive that even if I can't think of a use-case off the top of my head, there will eventually be a library author (if this proposal is accepted) who will have some cool idea that will require distinguishing these two scenarios. Like for example if this were used for a CLI-building library (think something like Typer) how the assignment to `None` could signify that it is an option with a default value of `None`, whereas the bare name would signify that it is a mandatory argument. Oh, and I think I've just discovered another thing that I'm not 100% sure I like. Even putting aside that I'm not a fan of decorators on the same line as the statement they are decorating (as I mentioned in an earlier response), you've got examples of variable decorators where no assignment is happening such as: @decorator var To me this breaks the symmetry between function decorators, which always decorate a function definition (an implicit form of assignment), and the proposed variable decorators. They are also confusing in the sense that the decorator is de-facto turning an otherwise invalid python statement legal. If you remove the decorator from the above example you will presumably get a `NameError`. I imagine this would then have to be special-cased somehow in the language spec so that an undefined name is not evaluated, but only when preceded by a decorator? I don't know, it seems messy to me. Also, I just can't quite see the value in them if I'm honest, whereas the version that is applied to an assignment statement: @decorator var: bool = True And even a bare type-hint version: @decorator var: bool seem to me to be far more self-evidently useful. On Thu, May 27, 2021 at 5:45 PM Ricky Teachey <ricky@teachey.org> wrote:

On Thu, May 27, 2021 at 1:40 PM Matt del Valle <matthewgdv@gmail.com> wrote:
...
Oh, and I think I've just discovered another thing that I'm not 100% sure I
Ok, agreed on all points. I think an eventual full-fledged proposal could easily put naked decorations like: @decorator var ...to the wayside, to be added later if people have a really good reason for it. --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler

On Thu, May 27, 2021 at 10:40 AM Matt del Valle <matthewgdv@gmail.com> wrote:
I agree that if we allow both then they should be distinguished somehow. They are different not just in value but in existence since on the next line down "var" will either be just fine or raise a NameError. If the decorator is expecting for some reason to reach into its parent scope and modify this variable, those are two very different situations.
I am confused why you are okay with @decorator var: bool but not @decorator var Yes, a bare name is currently an error while just a name and a type hint is valid, but the latter doesn't bind anything to the name, and using that identifier is still a NameError. So a decorator presumably can't return a value for either (or it could, but it would always be dropped). What could it do with a name and a type hint that is so much better than just a name? I am still very confused as to the scope of this counter proposal re variable decorating. I have only seen two such examples here @decorator variable # variable = decorator.__decoration_call__(None, "variable") @decorator variable = "spam" # variable = decorator.__decoration_call__(variable, "variable") But what is actually valid to follow a decorator in this proposal? Any simple expression, any expression? Is it limited to assignment espressions? Here are some interesting uses that were brought up in the other thread and I would like to know how they would work. @decorator spam = eggs = cheese = "tomatoes" @decorator spam, eggs, cheese = "tomatoes" @decorator spam, eggs = cheese = "tomatoes" @decorator spam = (eggs := "cheese) @decorator locals()[find_it() or "default"] = spam() Regards, ~Jeremiah

On Thu, May 27, 2021 at 10:25 AM Steven D'Aprano <steve@pearwood.info> wrote:
No, I understood the OP's proposal perfectly. I was agreeing with you implicitly when you previously said the inconsistency between the OP's proposal and current decorator is a problem: <STEVE WROTE>: On the other hand, it would be terribly confusing if the same syntax:
And furthermore, it is even a bigger problem than you explicitly made it out to be because of passing and object vs. passing a string representing the name of an object. So I was not illustrating the OP's proposal, but showing (but not proposing) a modified version of it that acts more like decorators today, so that we would have this: # NOT THE OP's PROPOSAL -- more consistent with decorators today, but not as quickly useful on its own @decorator("spam this") var # decorator("spam this")("var") ..rather than this: # OP's PROPOSAL @decorator("spam this") var # decorator("spam this", "var")
Yes, I am not proposing this behavior or saying it was the OP's, just illustrating to agree with you that it is so different from current decorator behavior, I don't think it should be considered (even though I sort of like how it looks).
Yes, it is looked up on the decorator, i.e., the result of the expression immediately to the right of the @ symbol. So it is only used when using *decorator syntax* (i.e., the @ symbol).
Yes.
I am sorry, I thought I made this clear early on in the proposal. yes, the decorator is the thing. More precisely, the result of the expression to the right of the @ symbol. By "any callable", I had in mind how any callable can be used as a decorator (so long as it isn't expecting more than 1 positional argument and no required kwd arguments). But really, the result any expression can be a decorator now (as of 3.10 I believe?), thought you might get an error: @1/2 def func(): ... # TypeError: 'float' object is not callable
Yes, I had written exactly that above the quoted part: So, for any existing callable with the name decorator, this:
Continuing on.
Invoking @flambé causes: flambé.__decoration_call__(Banana_Surprise, "Banana_Surprise") ...to be used, if flambé has that method.
HMMM.... to be totally honest I had not considered the problem this idea poses for introspection and documentation. But after giving it some thought, I don't see it as a significant problem. Yes, you would have to read the source code of flambé, or consult help/introspection tools that know about the new protocol, to know whether a __decoration_call__ method will be used. And introspection/help tools would need to present both the normal call signature: flambé() <---> flambé.__call__(self, spam, /, eggs, *args, cheese, **kwargs) ....and the decoration call signature, which is intended to be FIXED at two position args: @flambé <---> flambé.__decoration_call__(self, func, by_name) Of course, anyone could make the __decoration_call__ signature anything they wanted: @flambé <---> flambé.__decoration_call__(self, func, /, by_name, *chaos, pain="why?", **suffering) I do see that as a potential problem now; but I think it's a minor problem. Help and tutorial guides/files would have to teach people that there is potentially a different calling signature when decorator syntax is used, just as they teach that other syntactical symbols call different dunder methods when they are used: a += 1 a + 1 But I think the flexibility we will get could be worth this didactic overhead.
If you wrote the decorator, you know. If you didn't, why does it matter if it is being passed? Why do you need to know that? Some descriptor have __set_name__ methods, and others don't. We use them very happily in both cases. The callable function you are using as a decorator has a job; presumably you know what the job is or you'd not be using the callable. Do you really have to concern yourself with HOW it does it, unless you are the maintainer, or tracking down a bug inside the implementation of the decorator? At which point you'll have to read the source code anyway.
No. I am NOT-- repeat, NOT-- proposing a change to the behavior of the call operator, (). The call operator, (), ALWAYS does this: decorator_factory() <--> decorator_factory.__call__() This is true regardless of whether that call is prepended by the @ symbole. Clear? Under NO circumstances does: decorator_factory("spam this") ...result in this: decorator_factory.__decoration_call__("spam this") No. ONLY prepending the @ symbol at the beginning of a line invokes __decoration_call__, and it invokes it on the result of the expression immediately to the right, just as it does today: @decorator_factory() <--> decorator_factory().__decoration_call__ @( decorator := decorator_factory() ) <--> ( decorator := decorator_factory() ) .__decoration_call__ @tricky_decorator + 1 / 2 "lala" ^ None <--> ( tricky_decorator + 1 / 2 "lala" ^ None ).__decoration_call__ The object that is returned by the factory object is the decorator. NOT the decorator factory. These code blocks result in identical behavior for func: decorator = decorator_factory("spam this") @decorator def func(): ... @(decorator := decorator_factory("spam this")) def func(): ... @decorator_factory("spam this") def func(): ... The object returned by the expression to the right of the @ symbol is not in any way affected by the proposal. Everything happens as it does today. Zero changes.
I said it above already but I don't see why this is such a problem. Presumably you are using this callable as a decorator because you are confident to some degree it will do the job it is supposed to do. If it needs the by_name argument to do that job, it will make use of the __decoration_call__ dunder. If it doesn't need it, it won't.
No, as I explained above, that isn't the case. The decorator object returned by the factory call has two possible behaviors, yes. And yes, understand which happens, you have to look at how it is implemented.
No; it remains two: factory.__call__(*args, **kwargs) .__call__ (*more_args) .__decoration_call__ OR, the default: factory.__call__(*args, **kwargs) .__call__ (*more_args) .__call__
I don't blame you. The version of my proposal you had in mind was terrible; sorry I did not explain it so well the first time. I hope the version I actually have in mind will be better received. --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler

On Thu, May 27, 2021 at 9:34 AM Ricky Teachey <ricky@teachey.org> wrote:
I got the sense that people both liked reading my examples of same-line decorators and pushed back against not making them appear just like function decorators. One way to have my cake and you eat it too would be to relax the current decorator grammar to not require a NEWLINE. AFAICT there would be no ambiguity since after a decorator there must either be another "@" or a "def". Then both function and assignment decorating can be both. These would all be possible and not change the status quo. @cache def factorial(n): pass # This is your example @namedtuple Point = "x y z" # OP was @namedtuple("x y z") Point @not_null @indexed first_name: str
Actually the original proposal was @decorator("spam this") var # var = decorator("var", "spam this") The implied assignment that is not written was a big part of the proposal, as most of my examples were defining the name for the first time, and also the original proposal disallowed assignment (or any statement) after the decorator; it is only valid to have an identifier after. Also the "var" coming first is a subtle but important distinction. By providing the name as the first argument, all of my examples of callables currently in the standard library will work as you say out of the box. If it were to be passed in last, this new syntax would not be usable by any standard library callable (or even third party? Does anyone create factory functions that need the name and take it last?) and lots of new functions would have to be added. Regards, ~Jeremiah

On Thu, May 27, 2021 at 2:00 PM micro codery <ucodery@gmail.com> wrote:
...
By providing the name as the first argument, all
Yes that's true. I originally wrote the proposal with the arguments switched around. But I it seemed like it could be important to have the decorated_object argument appear first in the signatures for both __call__ and __decoration_call__, or it could cause confusion... However, I am open to the idea I was right the first time. If so, the default implementation at the top of the object food chain would be something like (using Matt's SENTINEL idea): def __decoration_call__(self, by_name, obj=SENTINEL): if func is SENTINEL: return self(by_name) return self(obj) On the one hand, this might be too expensive; an identity check and potentially two function calls occur for all @ decorations. And I don't see how to optimize this away with opcodes... On the other hand, how often are people invoking millions of @ decorations in a loop...? --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler

On Thu, May 27, 2021 at 12:33:20PM -0400, Ricky Teachey wrote:
Right: you can't tell from the code you are looking at whether flambé is being called as a new or old style decorator. You have to dig into the definition of flambé to work out the meaning of the code. On the one hand we have the current decorator protocol, where the object flambé is called with a single argument, the class Banana_Surprise. I can only think of three cases where something similar occurs, but the differences are critical. - Binary operators can call either or both of two reflected dunders, such as `__add__` and `__radd__`. - Iteration normally calls `__next__`, but it can fall back on the old sequence protocol that relies on `__getitem__`. - Augmented assignment calls the augmented assignment method, such as `__iadd__`, but may fall back on the regular `+` operator. In all of these cases, the intended semantics are the same and the arguments passed are (almost) the same. E.g. normally we would use the same signatures for each of the operators: def __add__(self, other): ... def __radd__(self, other): ... def __iadd__(self, other): ... and if fact for many purposes, add and radd will be the same function object. The old sequence protocol is a minor exception, because `__getitem__` requires an argument (it is called with 0, 1, 2, 3, ... until IndexError occurs). But that's almost an implementation detail. The fact that `__next__` takes no explicit enumeration, but `__getitem__` does, can be ignored when we think about the high-level concept of iteration. Conceptionally, at a high-level, the sequence and iterator protocols operate the same way: call the dunder repeatedly until an exception occurs and the details of which dunder and which exception don't matter. Iteration is iteration. But here, the details *do* matter. Old classic decorators and this proposed new decorator syntax do not share a high-level concept. There is a certain relationship in the sense that both the new and old protocols involve calling a function, but the details are very different: - old decorator protocol is a normal function call with a single argument; - new protocol uses a different method, with two arguments, one of which is implicit. Old decorator protocol is about *decorating* a function or class object with new functionality (wrapping it in a wrapper function, say, or modifying it in place). New decorator protocol and variable decoration doesn't decorate anything: there's nothing to decorate when used with a bare name: @decorate var which presumably evaluates to: var = decorate.__decorate_call__('var') So we're not decorating an object. There's no object to decorate! With a binding: @decorate var = expression var = decorate.__decorate_call__('var', expression) Again, conceptually we're not decorating anything. We're just reusing the `@` syntax for something that is not conceptually related to old style decorators. This is just a syntax for *implicitly* getting access to the left hand side assignment target name as a string. [...]
Because I don't know the meaning of the code. Am I decorating an object or doing something with the target name? I can't interpret the code until I know what protocol was supported by the decorator object. Let me give you an analogy: the `with` statement has a context manager protocol that uses an `__enter__` and `__exit__` pair of methods. With statements are awesome! They're so awesome that we should use the exact same "with" syntax for Pascal-style implicit attribute access, using a different protocol. https://docs.python.org/3/faq/design.html#why-doesn-t-python-have-a-with-sta... with myobject: do_something() with myotherobj: do_something() There is no visible difference between the two pieces of code, but one is an old style enter/exit context manager, the other is our hypothetical new style with syntax that does something completely different and calls different dunder methods. One of them is calling the global do_something and one is calling the object's do_something method. Which is which? Code that is different should look different. -- Steve

On Tue, Jun 1, 2021 at 9:57 PM Steven D'Aprano <steve@pearwood.info> wrote:
My understanding is that it would attempt to invoke __detonation_call__ (err, I mean, __decoration_call__, but just think how awesome the other would be) first, and if it can't find it, it falls back on regular __call__. That would parallel how most other things are done - repr falling back to str, iadd falling back to add, etc. That said, I still don't like the idea of decorating a bare name. A magic "assignment target" token that translates into the string form of the thing being assigned to is far more useful and far less magical. ChrisA

On Tue, Jun 01, 2021 at 10:17:32PM +1000, Chris Angelico wrote:
Sure, but the analogy is more like trying repr first to produce a string, and if that doesn't work, trying iter to produce a list. Function/class decoration shares little with "variable decoration" except the name and (proposed) use of the @ syntax. It seems to me that they are only the most loosely related concepts. Ricky's proposal to use a new dunder magnifies the differences and obfuscates the similarities.
Obviously I agree, given that I proposed a special @@ symbol to do that however Stéfane's real-world example of chaining variable decorators has forced me to take that idea seriously. @acl(READ, WRITE) @constraint(10 < _ < 100) @not_null @indexed @depends_on(whatever) @inject @... first_name: str https://mail.python.org/archives/list/python-ideas@python.org/message/DCKFMW... -- Steve

Another problem I have with this proposal is the new dunder method. Why do we need a new dunder? We're probably never going to "decorate" a variable and function with the same callable, the two use-cases are very different. But even if we do, we can easily distinguish the two cases. Let's dump the `__decorate_call__` dunder and just use normal call syntax. That will be a HUGE win for useability: any function, class or callable object can be used as the decorator, it doesn't have to be a custom object with a custom class that defines a special dunder. We can distinguish the two contexts by using different signatures. The signature used depends entirely on the call site, not the decorator, so it is easy for the interpreter to deal with. If the decorator is called on a function or class statement, a single argument is always passed, no exceptions: # always calls decorate with one argument @decorate def func(): # or class pass # --> func = decorate(func) If called on a variable, the number of arguments depends on whether it is a bare name, or a value and annotation are provided. There are exactly four cases: # bare name @decorate var # --> var = decorate('var') # name with annotation @decorate var: annot # --> var = decorate('var', annotation=annot) # name bound to value @decorate var = x # --> var = decorate('var', value=x) # name with annotation and value @decorate var: annot = x # --> var = decorate('var', annotation=annot, value=x) Keyword arguments are used because one or both of the value and the annotation may be completely missing. The decorator can either provide default values or collect keyword arguments with `**kwargs`. The only slightly awkward case is the bare variable case. Most of the time there will be no overlap between the function/class decorators and the bare variable decorator, but in the rare case that we need to use a single function in both cases, we can easily distinguish the two cases: def mydecorator(arg, **kwargs): if isinstance(arg, str): # must be decorating a variable ... else: # decorating a function or class assert kwarg == {} So it is easy to handle both uses in a single function, but I emphasise that this would be rare. Normally a single decorator would be used in the function/class case, or the variable case, but not both. -- Steve

On Tue, Jun 1, 2021 at 10:16 PM Steven D'Aprano <steve@pearwood.info> wrote:
I can't imagine any situation where you would *want* this, but it is actually possible for a decorator to be given a string:
This probably falls under "you shot yourself in the foot, so now you have a foot with a hole in it". ChrisA

On Tue, 1 Jun 2021 at 13:16, Steven D'Aprano <steve@pearwood.info> wrote:
I've yet to be convinced that variable annotations are sufficiently useful to be worth all of this complexity (and by "this" I mean any of the proposals being made I'm not singling out Steven's suggestion here). But if we do need this, I quite like the idea of making the distinction based on signature.
You don't need to do this. Just add another keyword argument "name": # bare name @decorate var # --> var = decorate(name='var') # name with annotation @decorate var: annot # --> var = decorate(name='var', annotation=annot) # name bound to value @decorate var = x # --> var = decorate(name='var', value=x) # name with annotation and value @decorate var: annot = x # --> var = decorate(name='var', annotation=annot, value=x) The single positional argument is reserved for function/class annotations, and will always be None for variable annotations. Paul

On 2021-05-26 09:43, Ricky Teachey wrote:
This seems contradictory to me. It looks like you're saying, "We shouldn't use decorator syntax to represent two different things (object vs name), but instead decorator syntax should give us access to two different things (object vs name)." I realize based on your proposal there is a distinction here but I think it's quite a narrow one and doesn't resolve the basic problem, which is that currently decorators operate on objects and these new proposals are about making them operate on names. I think there may be value in having some feature that lets us get access to the name side of an assignment. But I wouldn't call such a thing a "decorator", nor would I want to use the same @ syntax that is used for decorators. To me that would be confusing, because the behavior is totally different. Even with your __decorator_call__ proposal, there's still a jarring shift from, in some cases, using just the object, and in other cases stuffing a new parameter (the name) into the parameter list. That seems awkward to me. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown

Whoops, replying all this time. On Thu, May 27, 2021 at 2:32 PM micro codery <ucodery@gmail.com> wrote:
At this point, I have in mind any expression that appears to the right, which I believe is what is allowed today: @1/2 "lala" and money def func(): ...
I would also like to know how all of these work :D I am not sure about most of them but open to suggestions. The only one that I feel confident about is: @decorator spam = (eggs := "cheese) ...which, I think, should be: decorator.__decoration_call__(spam, "spam") Unfortunately for the proposal most people don't seem too thrilled with it. So I don't plan to spend a lot of time thinking through these examples and suggesting behavior. Anyone is welcome to do that though, this isn't MINE in the sense I am jealously guarding ownership of the details. :) On Thu, May 27, 2021 at 3:03 PM Brendan Barnwell <brenbarn@brenbarn.net> wrote:
Yes, and for this reason I really liked Steve's googly eyes proposal in the other thread. But I wonder if there value in specifically giving decorators access to the name side? It seems to me that it would open up a lot of possibilities, just as when descriptors learned their names. class Desc: def __set_name__(self, owner, name): self.name = name def __get__(self, instance, owner): if instance is None: pass print(f"I am { owner.__name__}.{self.name}") class C: v = Desc()
C().v I am C.v
We could make the __decoration_call__ method even more powerful. We could give it access not just to the name side, but to the type info, and even the code object/expression side (i.e., RHS). @decorator x: Fraction = 1/2 # decorator.__decoration_call__(x, "x", "1/2", Fraction) So, for example, a Math library can create the feature: @Math x: Fraction = 1/2 And x is: Fraction(1, 2) Yes, you can do: x = Fraction("1/2") ...today. I get that. But it's not as if this Math example is the ONLY thing it allows you to do. You also can easily avoid repeating things and making mistakes: @namedtuple_factory Point = "x y" I'm sure there are many other things I haven't thought of. --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler On Thu, May 27, 2021 at 3:03 PM Brendan Barnwell <brenbarn@brenbarn.net> wrote:

To reiterate my point from earlier in the thread, I am quite firmly opposed to having the decorator on the same line as the statement being decorated. I'll just copy-paste my reasoning for it: I like this:
And regarding your proposal to relax the newline requirement on function/class decorators: I got the sense that people both liked reading my examples of same-line
This would admittedly resolve the asymmetry between the proposed single-line variable decorator syntax and current decorators, but personally I just *really* don't like it. The only benefit is saving a line, but it comes at (in my opinion, your mileage may vary) a huge cost to legibility, and it goes against the zen: There should be one-- and preferably only one --obvious way to do it. And it's just something that has (almost) no precedent in python (aside from `async def`). It has the feel of something like: public static final native abstract void SomeMethod() {...} I'm imagining a future of reading python code like: @too @many("!") @decorators @on("a", "single", "line") def foo(...): And it makes me unhappy :( So going back to your original point of why I'm okay with decorating a bare type-hint, like: @decorate foo: int But not @decorate foo The reason is simply that if you take the decorator away the first one is legal, and the second one will raise `NameError`. I'm happy to decorate a statement that is valid on its own, but I'm against the idea of special-casing decorator syntax so that it can decorate otherwise-invalid statements. And I do think that there are legitimate uses for decorating a bare type-hint, since it does actually contain useful information the decorator might want to capture (for example, in one of its own instance attributes if the decorator is an object rather than a function, which it would have to be in order to implement __decoration_call__). You're right in that it wouldn't be able to return anything since no assignment is taking place, but there are still potential use-cases for it. I'll concede that there are also use-cases for decorating a completely naked name, but none of the ones I've seen so far seem to me compelling enough to break the rules like this. On Thu, May 27, 2021 at 8:43 PM Ricky Teachey <ricky@teachey.org> wrote:

On Thu, May 27, 2021 at 12:39 PM Ricky Teachey <ricky@teachey.org> wrote:
I don't think I was clear enough with my original question. I agree that any valid expression can follow the @, just like function decorators; I said as much in my original proposal. My question is what can follow after the line "@expression"? Could it be a with block; a lambda; a match statement? Probably just an assignment statement, but even just that covers a lot including multi-assignment or including a yield expression. But I think I know your answer to this from your reply to my later examples ;-) On Thu, May 27, 2021 at 2:19 PM Matt del Valle <matthewgdv@gmail.com> wrote:
Yes, there seems to have been a lot of rejection of the single-line decorator. I liked it for variable decorating because I didn't think developers would use it if it meant adding an extra line to every variable declaration that wanted to take advantage of the new syntax. Also, limiting its use to only identifiers kept the lines short. But I agree that it could lead some very unpythonic code, especially if multiple decorators on the same line were allowed. I just thought it looked better, but would you and others here like it better if the NEWLINE requirement was kept for all decorators? There is nothing in the original proposal that requires a single line. I also don't know what should happen for complicated assignments, and I think this has been the death of such variable decorator discussions in the past, so I would still push for only bare identifiers, with or without a type hint (but maybe it will be better received by more if the type hint is required?). I still think such a proposal is strong enough on its own to be of value to the language. So now the syntax would be: @decorator variable: Type @decorator("spam", eggs) variable: Type become variable = decorator("variable") variable = decorator("spam", eggs)("variable") I'm not sure that the type hint shouldn't be passed as an additional parameter, especially if one is always required. Yes it is still different from a function decorator in that you are not getting the variable object (which may or may not exist) but instead its name as a string. However, decorators that expect to be applied to variables should already be drastically different. Even if they did get the variable object, there would be not necessarily be a __code__ or __name__ or __dict__ attribute. They won't be callable (generally) where most current decorators attempt to call func, wrapped in an internal closure. In this case Stéfane's long example would still be valid, and some of the one-liners would instead look like this: class Colors(Enum): @str RED: str @str GREEN: str @str BLUE: str @namedtuple("x y z") Point: NamedTuple @os.getenv PATH: Optional[str] Regards, ~Jeremiah

On Fri, May 28, 2021 at 5:07 PM Rob Cliffe Co<rob.cliffe@btinternet.com> wrote:
Fair enough! If this became accepted syntax I would use it without type hints. Even for those of us that do use type hints in places, it shouldn’t generally be necessary because the decorator will have a return type annotation. I think the original argument was that currently bare identifiers are not allowed unless they have annotation. But this is introducing a new multiline syntax, and it makes no more sense to take away the second line and expect a naked decorator to be valid than it does remove the decorator and expect the naked identifier to be valid.

Ah, I think you might be missing the context of the original proposal? I do mean bare unbound identifiers - at lease as they occur in this new syntax. # currently works spam = “eggs” spam: eggs # currently a NameError spam # proposed to work, currently illegal @spam eggs @spam(“eggs”) cheese @spam eggs: str But none of this would change the first three examples.

I think there's a fundamental difference between your original proposal and the OPs proposal in this thread, which is that you seem to envision variable decorators as granting access to just a name (and potentially a type-hint?), whereas the OP wants the decorators extended to assignment in general. Basically, your proposal is a subset of the OPs proposal. Any implementation of the OPs decorators would also allow us to do all the things you want to be able to do with variable decorators (as a happy side-effect), but would also offer a LOT more functionality. My view is that variable decorators should be legal for: 1) assignment statements (so that the decorator can modify or replace the object that is getting assigned) 2) bare type-hints, since these are arguably *a sort of* assignment operation that still carries useful info a decorator might want to capture (you're not assigning the name and its value to globals(), but rather the name and its type to globals()['__annotations__']) If there was huge pushback against point 2 I'd be happy to see it deferred and revisit it at a later point, but as far as I'm concerned point 1 is what actually interests me about this proposal. Your point about complicated assignment being a problem seems solvable using the following rules: 1) Variable decorators only capture the name (or names) on the LHS of a true assignment statement (or bare type-hint). Expressions that happen to bind names (such as the walrus operator, for loop, etc.) don't have their names available for use in the decorator. 2) The proposed __decoration_call__ is given the value, name, and type-hint from the decorated statement (and potentially the code object from the RHS, as OP suggested). We could provide the names as a series of useful pre-parsed parsed AST-like objects, like: @decorate some_obj.foo = first, _, third, *some_obj.rest = some_dict[(lost_name := ('ba' + 'r'))] = baz = (another_lost_name := [0, 1, 2, 3, 4]) such that __decoration_call__ receives: def __decoration_call__(self, obj, names, annotation): print(obj) # [0, 1, 2, 3, 4] print(names) # ParsedNames(names=[ # ObjectAssignment(obj={reference_to_some_obj}, name='foo'), # DestructuringAssignment(names=[ # BasicAssignment(name='first'), # BasicAssignment(name='_'), # BasicAssignment(name='third'), # ObjectAssignment(obj={reference_to_some_obj}, name='*rest') # ]), # ItemAssignment(obj={ref_to_some_dict}, name='bar'), # BasicAssignment(name='baz') # ]) print(annotation) # NOTSET Notice that as per point 1 above, `lost_name` and `another_lost_name` do not get captured, because they are not part of the assignment statement. They're only sub-expressions. Also, notice that expressions like 'ba' + 'r' are evaluated before being passed to the decorator, so that it actually receives 'bar' for the __setitem__ assignment name. 3) Augmented assignment gets expanded out to its true meaning before being passed to the decorator, so for example: foo = 3 @decorate foo += 2 behaves exactly the same as: foo = 3 @decorate foo = foo + 2 If there are still any issues given these rules that I can't think of I'd be happy for people to raise them. There are so many edge-cases in python assignment that I could very easily be forgetting about something. But I think this covers 99% of cases. class Colors(Enum):
With decorated assignments these examples could instead look something like: class Color(Enum): @string_values RED, GREEN, BLUE = auto() Here the decorator would return a tuple like: ('RED', 'GREEN', 'BLUE'), which would be assigned instead of whatever is actually on the RHS And for the namedtuple: @namedtuple Point = 'x', 'y', 'z' The third one would look just like you suggested. On Sat, May 29, 2021 at 3:28 AM micro codery <ucodery@gmail.com> wrote:

On Sat, May 29, 2021 at 12:46:11PM +0100, Matt del Valle wrote:
Can you explain what benefit a variable decorator in this case would have over a function call? @func var = expression is, I expect, the same as: var = func(expression) or with the assignment target magic: var = func('var', expression)
Once we move out of bare type hints into calling a function, it's not a bare type hint any more. @func var: int if it calls func, is not the same as var: int which has no runtime effect. A note on augmented assignment:
3) Augmented assignment gets expanded out to its true meaning before being passed to the decorator, so for example:
The true meaning of augmented assignment is not to expand the line of code out to assignment with the operator, but a method call: target += expression # not this target = target + expression # but this instead: target = type(target).__iadd__(target, expression) Only if `__iadd__` doesn't exist does the interpreter fall back on the plus operator. This is why classes can implement augmented assignment as inplace operators: L = [1, 2, 3] L += [4] # modifies L in place L = L + [4] # creates a new list and assigns to L This is a significant difference as soon as you have multiple references to the original list. -- Steve
participants (9)
-
Brendan Barnwell
-
Chris Angelico
-
Matt del Valle
-
micro codery
-
Paul Moore
-
Ricky Teachey
-
Rob Cliffe
-
Steven D'Aprano
-
Valentin Berlier