Decorators on variables
Variable decorators have been suggested here before, as have new statements that could also achieve the same level of access to the binding name. However I propose a much more restricted syntax that would make for less edge cases where what is actually passed to the decorator callable may be ambiguous. Basically this would add syntax to python that would transform @decorator("spam this") variable into variable = decorator("variable", "spam this") The decorator would be limited to being on the same line as the variable name. Additionally, the variable can not also be assigned to on the same line, so @decorator("spam this") variable = eggs would be a SyntaxError. Annotating these variable would be allowed @decoratopr("spam this") variable: Food The actual decorator would follow the same rules as current decorators in that the parentheses can be omitted if no other arguments are required to the function and any arbitrary expression can be used, however the same restraint should be cautioned as is for current decorators post PEP 614. The most straightforward use of this decorating would be that str() could be used without change to declare variables that are strings of the same name, generally cutting down on repeating yourself when declaring constants. #instead of GREEN = "GREEN" @str GREEN The classic example of a factory that needs its target name as a string value is namedtuple. I don't think this example is enough alone to sway this list but I bring it up as it has also been at the front of past ideas to allow variable decorating. #instead of Point = namedtuple("Point", "x y z") @namedtuple("x y z") Point I have sometimes heard that dataclasses make this no longer an issue, implying that dataclasses can be used in place of namedtuple in any new code. Putting aside that dataclasses are not a strict replacement for namedtuple, the dataclasses module continues this factory design with the make_dataclass function. #instead of Point = make_dataclass("Point", [("x", int), ("y", int), ("z", int)]) @make_dataclass([("x", int), ("y", int), ("z", int)]) Point Enum is one more case where there is a factory equivalent to the class declaration. #instead of Colors = Enum("Colors", "RED GREEN BLUE") @Enum("RED GREEN BLUE") Colors If you want each enum member value to match its name there is even more repetition, however by using the original example for decorated variables, this can be reduced when using the more common class style. class Colors(Enum): @str RED @str GREEN @str BLUE One final standard library module that I will mention because it seems to keep adding similar factory functions is typing. It already has the factory functions NewType, TypeVar, ParamSpec, TypedDict, NamedTuple. Their use with the new variable decorator would look much the same as earlier examples, but here would be one case where type hinting the variable in such a statement could aid the decorator function #instead of UUIDType = NewType("UUIDType", str) @NewType UUIDType: str If adopted, no change to these functions would be necessary to allow their use as a variable decorators as they all already take the name as a string in the first argument. Additionally, as variable decorators are actually callables just like the current decorators, no new restrictions are being forced on users of the language. If a user really wants to create a namedtuple with a typename of Foo but assign the result to a variable named ___trippleS33cret___name__here_ the language will not stop them. I would very much like to know if any module maintainers are interested in this syntax. Convincing arguments from the standard lib (whether you consider the above convincing or not) are probably not enough to get a syntax change into python. But if enough third party modules want to use this functionality I believe that would be the real push to get it done. Specifically this idea first started growing when I read on another ideas message that using the module sympy would often start with a line like x = symbol('x') which, with no change, could be used as a variable decorator @symbol x Now what may be even more typical (I really have never used the module myself) is defining all needed symbols on one line with x, y, z = symbol('x y z') which could conceivably be rewritten like @symbol x, y, z However I have not yet been able to decide if multi-assignment would be overall beneficial to this new syntax. Probably not, at least at this stage. It would not be hard for a decorator function to return multiple values, but how would the multiple names be passed in? A string with whitespace like this original, and namedtuple's field_names, is possible as no valid identifier can have a space in it. However, it forces any would-be variable decorator to know this and check if a space is in its first argument, even if it cannot produce multiple values. Another option is to pass in an iterable, but the same additional effort would be placed on decorators that otherwise don't want to deal with multi-assignment. Unpacking the names as the first n arguments would mean that decorators that just want one variable can get just that and those that may want more will also get them, but then practically no variable decorator could take additional non-keyword arguments. Regards ~ Jeremiah
I don't have much of substance to say other than this proposal really made me go "oh I like that" several times. There may be downsides/arguments against I'm not considering, but I look forward to following the conversation and hope it gets a serious hearing. On Mon, May 24, 2021, 9:38 PM micro codery <ucodery@gmail.com> wrote:
Variable decorators have been suggested here before, as have new statements that could also achieve the same level of access to the binding name. However I propose a much more restricted syntax that would make for less edge cases where what is actually passed to the decorator callable may be ambiguous.
Basically this would add syntax to python that would transform @decorator("spam this") variable into variable = decorator("variable", "spam this")
The decorator would be limited to being on the same line as the variable name. Additionally, the variable can not also be assigned to on the same line, so @decorator("spam this") variable = eggs would be a SyntaxError. Annotating these variable would be allowed @decoratopr("spam this") variable: Food The actual decorator would follow the same rules as current decorators in that the parentheses can be omitted if no other arguments are required to the function and any arbitrary expression can be used, however the same restraint should be cautioned as is for current decorators post PEP 614.
The most straightforward use of this decorating would be that str() could be used without change to declare variables that are strings of the same name, generally cutting down on repeating yourself when declaring constants.
#instead of GREEN = "GREEN" @str GREEN
The classic example of a factory that needs its target name as a string value is namedtuple. I don't think this example is enough alone to sway this list but I bring it up as it has also been at the front of past ideas to allow variable decorating.
#instead of Point = namedtuple("Point", "x y z") @namedtuple("x y z") Point
I have sometimes heard that dataclasses make this no longer an issue, implying that dataclasses can be used in place of namedtuple in any new code. Putting aside that dataclasses are not a strict replacement for namedtuple, the dataclasses module continues this factory design with the make_dataclass function.
#instead of Point = make_dataclass("Point", [("x", int), ("y", int), ("z", int)]) @make_dataclass([("x", int), ("y", int), ("z", int)]) Point
Enum is one more case where there is a factory equivalent to the class declaration.
#instead of Colors = Enum("Colors", "RED GREEN BLUE") @Enum("RED GREEN BLUE") Colors
If you want each enum member value to match its name there is even more repetition, however by using the original example for decorated variables, this can be reduced when using the more common class style.
class Colors(Enum): @str RED @str GREEN @str BLUE
One final standard library module that I will mention because it seems to keep adding similar factory functions is typing. It already has the factory functions NewType, TypeVar, ParamSpec, TypedDict, NamedTuple. Their use with the new variable decorator would look much the same as earlier examples, but here would be one case where type hinting the variable in such a statement could aid the decorator function
#instead of UUIDType = NewType("UUIDType", str) @NewType UUIDType: str
If adopted, no change to these functions would be necessary to allow their use as a variable decorators as they all already take the name as a string in the first argument. Additionally, as variable decorators are actually callables just like the current decorators, no new restrictions are being forced on users of the language. If a user really wants to create a namedtuple with a typename of Foo but assign the result to a variable named ___trippleS33cret___name__here_ the language will not stop them.
I would very much like to know if any module maintainers are interested in this syntax. Convincing arguments from the standard lib (whether you consider the above convincing or not) are probably not enough to get a syntax change into python. But if enough third party modules want to use this functionality I believe that would be the real push to get it done.
Specifically this idea first started growing when I read on another ideas message that using the module sympy would often start with a line like x = symbol('x') which, with no change, could be used as a variable decorator @symbol x Now what may be even more typical (I really have never used the module myself) is defining all needed symbols on one line with x, y, z = symbol('x y z') which could conceivably be rewritten like @symbol x, y, z However I have not yet been able to decide if multi-assignment would be overall beneficial to this new syntax. Probably not, at least at this stage. It would not be hard for a decorator function to return multiple values, but how would the multiple names be passed in? A string with whitespace like this original, and namedtuple's field_names, is possible as no valid identifier can have a space in it. However, it forces any would-be variable decorator to know this and check if a space is in its first argument, even if it cannot produce multiple values. Another option is to pass in an iterable, but the same additional effort would be placed on decorators that otherwise don't want to deal with multi-assignment. Unpacking the names as the first n arguments would mean that decorators that just want one variable can get just that and those that may want more will also get them, but then practically no variable decorator could take additional non-keyword arguments.
Regards ~ Jeremiah _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/2FOIN5... Code of Conduct: http://python.org/psf/codeofconduct/
On 5/24/21 6:36 PM, micro codery wrote:
Variable decorators have been suggested here before, as have new statements that could also achieve the same level of access to the binding name. However I propose a much more restricted syntax that would make for less edge cases where what is actually passed to the decorator callable may be ambiguous.
#instead of GREEN = "GREEN" @str GREEN
#instead of Point = namedtuple("Point", "x y z") @namedtuple("x y z") Point
#instead of Point = make_dataclass("Point", [("x", int), ("y", int), ("z", int)]) @make_dataclass([("x", int), ("y", int), ("z", int)]) Point
#instead of Colors = Enum("Colors", "RED GREEN BLUE") @Enum("RED GREEN BLUE") Colors
class Colors(Enum): @str RED @str GREEN @str BLUE
#instead of UUIDType = NewType("UUIDType", str) @NewType UUIDType: str
I think that looks very interesting. -- ~Ethan~
On Mon, May 24, 2021 at 06:36:47PM -0700, micro codery wrote:
Basically this would add syntax to python that would transform @decorator("spam this") variable into variable = decorator("variable", "spam this")
That is confusingly different from decorator syntax in other contexts. Your proposal appears to be: @decorator(expression) targetname # transformed into: targetname = decorator("targetname", expression) But in class and function decorator contexts, the equivalent syntax is equivalent to: @decorator("spam this") def func(): pass # transformed to: def func(): pass func = decorator("spam this")(func) except that in CPython it is not actually implemented as two separate steps like that. But the critical difference is that the argument "spam this" should be passed to the decorator *factory*, which then returns the actual decorator that gets applied to the variable. (Or function/ class in the case of regulator decorator syntax.) To my mind, the only interesting part of this proposal is access to the binding target name. If we remove that from the proposal, variable decorators are nothing more than function calls, and we already can do that: variable = decorator("spam this") Wanting the name of the binding target doesn't come up often, but when it does, it's often very useful for the right-hand side of the assignment to know what the left hand side is, as a string, but the only way to do so is to manually provide it: # A trivial example. myclass = type("myclass", bases, namespace) # A common example. RED = "RED" Here's a counter-proposal: we have a special symbol which is transformed at compile-time to the left hand assignment target as a string. Let's say we make that special expression `@@` or the googly-eyes symbol. Then: RED = @@ # transforms to `RED = 'RED'` GREEN = "dark " + @@.lower() # transforms to `GREEN = "dark " + 'GREEN'.lower()` myclass = type(@@, bases, namespace) # transforms to `myclass = type('myclass', bases, namespace)` # Not all functions expect the name as first argument. result = function(arg, value, @@) # transforms to `result = function(arg, value, 'result')` If there's no target, it resolves to None: print(@@) # print(None) or if people prefer a SyntaxError, I'm okay with that too. Targets aren't limited to a single bare name. spam.eggs = @@ # spam.eggs = 'spam.eggs' mylist[2] = @@ # mylist[2] = 'mylist[2]' If the key or index is not known at compile-time, it is a syntax error: mylist[getindex()] = @@ # SyntaxError Chained assignments transform to a tuple of target names: spam = eggs = cheese = func(arg, @@) # spam = eggs = cheese = func(arg, ('spam', 'eggs', 'cheese')) Sequence unpacking assignment gets transformed as a single comma-seperated string: spam.eggs, foo, *bar = func(arg, @@) # spam.eggs, foo, *bar = func(arg, ('spam.eggs,foo,*bar')) This would be good for sympy: a, b, c, d, w, x, y, z = sympy.symbols(@@) Target resolution is performed at compile-time, not runtime. There's no global variable called "@@". That means that this won't work: code = compile("type(@@, bases, namespaces)", '', 'single') # above transforms to `type(None, bases, namespace)` myclass = eval(code) But I think that restriction is fine. -- Steve
On Tue, May 25, 2021 at 5:29 PM Steven D'Aprano <steve@pearwood.info> wrote:
Here's a counter-proposal: we have a special symbol which is transformed at compile-time to the left hand assignment target as a string. Let's say we make that special expression `@@` or the googly-eyes symbol.
This is sounding promising. I'm liking this.
Then:
RED = @@ # transforms to `RED = 'RED'`
GREEN = "dark " + @@.lower() # transforms to `GREEN = "dark " + 'GREEN'.lower()`
myclass = type(@@, bases, namespace) # transforms to `myclass = type('myclass', bases, namespace)`
# Not all functions expect the name as first argument. result = function(arg, value, @@) # transforms to `result = function(arg, value, 'result')`
Decent.
If there's no target, it resolves to None:
print(@@) # print(None)
or if people prefer a SyntaxError, I'm okay with that too.
Bikesheddable. I'd be inclined to go with SyntaxError, and make @@ a syntactic construct that is inherently part of the assignment; that way, there's no confusion in other contexts. No big deal either way.
Targets aren't limited to a single bare name.
spam.eggs = @@ # spam.eggs = 'spam.eggs'
mylist[2] = @@ # mylist[2] = 'mylist[2]'
What about: mylist[ 2 ] = @@ ? I'm inclined to say that the assignment target is reconstructed from the AST, in order to make it consistent (so this would also assign the string 'mylist[2]'). But, again, bikesheddable.
If the key or index is not known at compile-time, it is a syntax error:
mylist[getindex()] = @@ # SyntaxError
Also bikesheddable; I'd actually say that this should assign the string 'mylist[getindex()]', regardless of the value returned by getindex.
Chained assignments transform to a tuple of target names:
spam = eggs = cheese = func(arg, @@) # spam = eggs = cheese = func(arg, ('spam', 'eggs', 'cheese'))
Hmm. Everything else gives you a single string, this one doesn't. I'd actually be inclined to switch around this one and the next one...
Sequence unpacking assignment gets transformed as a single comma-seperated string:
spam.eggs, foo, *bar = func(arg, @@) # spam.eggs, foo, *bar = func(arg, ('spam.eggs,foo,*bar'))
... so that assigning the same thing to multiple names gives you a space-separated string (or equals-separated, "spam=eggs=cheese"), but unpacking gives you a tuple of targets, since it then nicely parallels the result it's expecting from the function. That would mean that: # This assigns a single string to them all eg "spam eggs cheese" spam = eggs = cheese = @@ # This assigns a string to each one: spam, eggs, cheese = @@ # and is equivalent to: spam = @@; eggs = @@; cheese = @@
Target resolution is performed at compile-time, not runtime. There's no global variable called "@@". That means that this won't work:
code = compile("type(@@, bases, namespaces)", '', 'single') # above transforms to `type(None, bases, namespace)` myclass = eval(code)
But I think that restriction is fine.
Absolutely. This has the same sort of value as the C preprocessor stringification operator. It's incredibly handy in making self-referential statements. Python has places where that happens by magic (class and function definitions), but if you want to create your own function that gets the same benefit.... well.... there's __set_name__ if your thing gets put into a class, but otherwise you have to repeat the name. Questions: 1) Is this restricted to the "=" assignment operator, or will other operators trigger this too? x += f(@@) # ? if x := f(@@): # ? 2) What about other forms of assignment? for spam in foo(@@): # ? with open(@@ + ".json") as config: # ? from sys import @@ as argv # okay that's just stupid 3) Is this a string literal, or a magic token that happens to evaluate as a string? x = @@ ".json" # Legal if @@ is a string literal No wrong answers. (Well, unless you say "tomato". That is a very wrong answer to a yes/no question.) I'm liking this. It might mean that class syntax and decorator abuse become less necessary as ways to get around name duplication. ChrisA
On Tue, May 25, 2021 at 8:11 AM Chris Angelico <rosuav@gmail.com> wrote:
On Tue, May 25, 2021 at 5:29 PM Steven D'Aprano <steve@pearwood.info> wrote:
Here's a counter-proposal: we have a special symbol which is transformed at compile-time to the left hand assignment target as a string. Let's say we make that special expression `@@` or the googly-eyes symbol.
...
Questions:
1) Is this restricted to the "=" assignment operator, or will other operators trigger this too? x += f(@@) # ? if x := f(@@): # ?
2) What about other forms of assignment? for spam in foo(@@): # ? with open(@@ + ".json") as config: # ? from sys import @@ as argv # okay that's just stupid
3) Is this a string literal, or a magic token that happens to evaluate as a string? x = @@ ".json" # Legal if @@ is a string literal
No wrong answers. (Well, unless you say "tomato". That is a very wrong answer to a yes/no question.)
I'm liking this. It might mean that class syntax and decorator abuse become less necessary as ways to get around name duplication.
ChrisA
Another one to add to the list: does it work with lambdas on the RHS? Does it matter how nested? x = lambda: @@ y = lambda: lambda: lambda: @@ --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler
On Tue, May 25, 2021 at 8:11 AM Chris Angelico <rosuav@gmail.com> wrote:
On Tue, May 25, 2021 at 5:29 PM Steven D'Aprano <steve@pearwood.info> wrote:
Here's a counter-proposal: we have a special symbol which is transformed at compile-time to the left hand assignment target as a string. Let's say we make that special expression `@@` or the googly-eyes symbol.
This is sounding promising. I'm liking this.
...
Chained assignments transform to a tuple of target names:
spam = eggs = cheese = func(arg, @@) # spam = eggs = cheese = func(arg, ('spam', 'eggs', 'cheese'))
Hmm. Everything else gives you a single string, this one doesn't. I'd actually be inclined to switch around this one and the next one...
Sequence unpacking assignment gets transformed as a single comma-seperated string:
spam.eggs, foo, *bar = func(arg, @@) # spam.eggs, foo, *bar = func(arg, ('spam.eggs,foo,*bar'))
... so that assigning the same thing to multiple names gives you a space-separated string (or equals-separated, "spam=eggs=cheese"), but unpacking gives you a tuple of targets, since it then nicely parallels the result it's expecting from the function. That would mean that:
# This assigns a single string to them all eg "spam eggs cheese" spam = eggs = cheese = @@ # This assigns a string to each one: spam, eggs, cheese = @@ # and is equivalent to: spam = @@; eggs = @@; cheese = @@
...
No wrong answers. (Well, unless you say "tomato". That is a very wrong answer to a yes/no question.)
I'm liking this. It might mean that class syntax and decorator abuse become less necessary as ways to get around name duplication.
ChrisA
(In the spirit of no wrong answers!) One of the motivating examples from the OP's post was symbolic math. As such I think this looks pretty nice (side note: in sympy the sympy.Symbol factory name is symbols, not symbol): # x = symbols("x") @symbols x For a single symbol, it would be this using Steven D's googly-eyes counter proposal, which also looks good to me: # x = symbols("x") x = symbols(@@) I think Steven makes a good case that the googly-eyes brings with it the best part of the OP's proposal, and avoids inconsistent, probably confusing decorator behavior (compared to function decorators). Continuing on with Steven's counter proposal and the motivating example of symbolic math, for multiple symbols it would be this: # x, y, z = symbols( 'x,y,z' ) x, y, z = symbols(@@) However, even though it works for the symbols example, this meaning looks odd to me: # x, y, z = symbols( 'x,y,z' ) ...it seems like this should in fact be *three separate assignment operations*, not a single assignment operation. So, instead: x, y, z = symbols(@@) # three assignments, not one (and this is Chris A's suggestion!): # x = symbols( 'x' ); y = symbols( 'y' ); z = symbols( 'z' ) This version also works for the symbols example, though it calls the factory function 3 times rather than 1. Is that an important downside? I am on the fence for the better meaning of this: x = y = z = symbols(@@) # x = y = z = symbols( ('x', 'y', 'z') ) # Steven's -- produces (Symbol('x'), Symbol('y'), Symbol('z')) ) # x = y = z = symbols( 'x y z' ) # Chris' version 1 -- produces (Symbol('x'), Symbol('y'), Symbol('z')) # x = y = z = symbols( 'x=y=z' ) # Chris' version 2 -- produces Symbol('x=y=z') As noted above, for all three versions if a user tried this it may appear to work at first (no immediate errors) but in actuality, x, y, and z would almost certainly not contain what the user wants. This seems potentially frustrating? I note that the symbols example is not the be-all end-all example, but going through it was instructive for me on what the differences of Steven's and Chris' suggestions are. And maybe other examples would behave similarly? Unsure. --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler
Ricky Teachey writes:
Continuing on with Steven's counter proposal and the motivating example of symbolic math, for multiple symbols it would be this:
# x, y, z = symbols( 'x,y,z' ) x, y, z = symbols(@@)
However, even though it works for the symbols example, this meaning looks odd to me:
# x, y, z = symbols( 'x,y,z' )
...it seems like this should in fact be *three separate assignment operations*, not a single assignment operation.
I don't think so. AFAIK, that syntax is not three separate assignment statments, conceptually: it's a tuple construction followed by destructuring assignment. That is, it's the problem of symbols() to return a compatible tuple.
This version also works for the symbols example, though it calls the factory function 3 times rather than 1. Is that an important downside?
Even if we interpret it as three assignments, I don't think so. Internally, it has to call Symbol() three times anyway, and you save a @@.split() call. The other possibility would be to interpret x, y, z = symbols(@@) as x, y, z = symbols('x', 'y', 'z') SymPy can change this API, or add a new one, to be compatible with the googly eyes feature. The hardest part would be coming up with a name if a new API was the route chosen. SymPy users would adopt by acclamation, I think. The problem with this is that this doesn't work for anything but sequences, although I'm not sure how useful most other expressions on the lhs are anyway.
I am on the fence for the better meaning of this:
x = y = z = symbols(@@)
I think this already has a meaning: x = y = (z := symbols(@@)) You could change it, of course, but I'd WTF every time I'm afraid. For the symbols() application, I'm much more sympathetic to the x, y, z = symbols(@@) syntax, most likely with the str value of @@ constructed in a canonical way from the AST as Chris suggests. SymPy can adapt or not as it chooses. Bikeshed: we could put a nail in the coffin of "from __future__ import barry_as_flufl" using "<>" as the RHS token instead of googly eyes. (0.5 wink -- I realized the implication for barry_as_flufl only after thinking of this alternative.) I'd kinda like to save @@ for some hypothetical matrix operation. Steve
While I am not at this point dropping the idea of @decorator(expression) targetname a thought occurred to me yesterday after discussing targetname as a string in assignments (and I appreciate the continued discussion in a new thread but this is a new counter proposal so keeping it here). There is at least one other place where python saves us from retyping the lhs in an assignment statement: augmented assignment. It is not being used as a string, true, and it's much more difficult to introduce a subtle bug by mis-typing the name if one were to use the standard operator instead. But we still have augmented assignment for all operators IMO because programmers are productively lazy and came up with a way to not retype their variable. What if, instead of adding a new magic symbol to the assignment statement (statements? still not sure which ones are supported in the counter proposal) we add a new assignment operator and corresponding dunder method that would perform the real operation behind the syntactic sugar. For now I propose the operator "<==" and dunder "__assign__". Both very bikeshedable, but allows us to draw some examples RED <== str would become RED = str.__assign__("RED") # or is this how it's actually done? don't remember # RED = type(str).__assign__(str, "RED") where str now has the method def __assign__(cls, assignment_name): return assignment_name Following the example of augmented assignment, both of these would be SyntaxErrors: spam = eggs <== str spam, eggs <== str Although there is probably a way to make the second one work, that can be experimented with in a reference implementation or saved for a later PEP. Some more of my original examples with the new operator: Point <== namedtuple("x y z") UUIDType: str <== NewType Colors <== Enum("RED GREEN BLUE") class Colors(Enum): RED <== str GREEN <== str BLUE< == str It would also be possible to reuse a factory, similar to the way a decorator that takes arguments first creates a sort of closure and is then that is passed the function object in a subsequent call. SPAM_DB <== get_sql_db(pass=_secret, user=user) TABLE1 <== SPAM_DB TABLE2 <== SPAM_DB would be the same as SPAM_DB = get_sql_db(pass=_secret, user=user).__assign__("SPAM_DB") TABLE1 = SPAM_DB.__assign__("TABLE1") TABLE2 = SPAM_DB.__assign__("TABLE2") Regards ~Jeremiah
On Tue, May 25, 2021 at 10:10:12PM +1000, Chris Angelico wrote:
On Tue, May 25, 2021 at 5:29 PM Steven D'Aprano <steve@pearwood.info> wrote:
Here's a counter-proposal: we have a special symbol which is transformed at compile-time to the left hand assignment target as a string. Let's say we make that special expression `@@` or the googly-eyes symbol.
[...]
Targets aren't limited to a single bare name.
spam.eggs = @@ # spam.eggs = 'spam.eggs'
mylist[2] = @@ # mylist[2] = 'mylist[2]'
What about:
mylist[ 2 ] = @@
? I'm inclined to say that the assignment target is reconstructed from the AST, in order to make it consistent (so this would also assign the string 'mylist[2]'). But, again, bikesheddable.
Let the implementation decide whether it is easier to get the target from the source code or the AST. I don't care either way.
If the key or index is not known at compile-time, it is a syntax error:
mylist[getindex()] = @@ # SyntaxError
Also bikesheddable; I'd actually say that this should assign the string 'mylist[getindex()]', regardless of the value returned by getindex.
My reasoning it that it is safer and more conservative to start with a restriction and relax it later, than to start with a less restrictive version and regret it. But I could be persuaded otherwise :-)
Chained assignments transform to a tuple of target names:
spam = eggs = cheese = func(arg, @@) # spam = eggs = cheese = func(arg, ('spam', 'eggs', 'cheese'))
Hmm. Everything else gives you a single string, this one doesn't. I'd actually be inclined to switch around this one and the next one...
A complication I just thought of is that you can have chained assignment within a sequence unpacking assignment: spam, eggs, aardvark = foo = bar, hovercraft = 'abcd' and the other way around: spam = eggs = (aardvark, foo, bar) = hovercraft = 'abc' That's going to make things tricky.
Sequence unpacking assignment gets transformed as a single comma-seperated string:
spam.eggs, foo, *bar = func(arg, @@) # spam.eggs, foo, *bar = func(arg, ('spam.eggs,foo,*bar'))
... so that assigning the same thing to multiple names gives you a space-separated string (or equals-separated, "spam=eggs=cheese"), but unpacking gives you a tuple of targets, since it then nicely parallels the result it's expecting from the function. That would mean that:
# This assigns a single string to them all eg "spam eggs cheese" spam = eggs = cheese = @@ # This assigns a string to each one: spam, eggs, cheese = @@ # and is equivalent to: spam = @@; eggs = @@; cheese = @@
I have to think about that some more :-)
This has the same sort of value as the C preprocessor stringification operator. It's incredibly handy in making self-referential statements.
Nice analogy.
Python has places where that happens by magic (class and function definitions), but if you want to create your own function that gets the same benefit.... well.... there's __set_name__ if your thing gets put into a class, but otherwise you have to repeat the name.
Yes.
Questions:
1) Is this restricted to the "=" assignment operator, or will other operators trigger this too? x += f(@@) # ? if x := f(@@): # ?
I hadn't thought that far ahead. I did think that we ought to exclude the walrus operator because it would be ambiguous: spam = eggs * (cheese:=foo+bar(@@)) Does @@ get the value 'cheese' or 'spam'? If we require an assignment statement, then it can only be 'spam' and the ambiguity is gone.
2) What about other forms of assignment? for spam in foo(@@): # ?
YAGNI. We can always extend the functionality later. Let's keep it simple: it works for assignment statements, not every binding operation.
3) Is this a string literal, or a magic token that happens to evaluate as a string?
An actual string.
x = @@ ".json" # Legal if @@ is a string literal
Heh, I wouldn't necessarily require that. (Nor would I object to it.) Implicit string concatenation is a nice feature, but I'm not sure we want to extend it. Its not hard to slot an explicit `+` in there. -- Steve
On Wed, May 26, 2021 at 8:44 PM Steven D'Aprano <steve@pearwood.info> wrote:
On Tue, May 25, 2021 at 10:10:12PM +1000, Chris Angelico wrote:
On Tue, May 25, 2021 at 5:29 PM Steven D'Aprano <steve@pearwood.info> wrote:
Here's a counter-proposal: we have a special symbol which is transformed at compile-time to the left hand assignment target as a string. Let's say we make that special expression `@@` or the googly-eyes symbol.
[...]
Targets aren't limited to a single bare name.
spam.eggs = @@ # spam.eggs = 'spam.eggs'
mylist[2] = @@ # mylist[2] = 'mylist[2]'
What about:
mylist[ 2 ] = @@
? I'm inclined to say that the assignment target is reconstructed from the AST, in order to make it consistent (so this would also assign the string 'mylist[2]'). But, again, bikesheddable.
Let the implementation decide whether it is easier to get the target from the source code or the AST. I don't care either way.
Fair enough. This sort of thing would need to be settled before a PEP could be accepted, but by then, there'll want to be a reference implementation.
Chained assignments transform to a tuple of target names:
spam = eggs = cheese = func(arg, @@) # spam = eggs = cheese = func(arg, ('spam', 'eggs', 'cheese'))
Hmm. Everything else gives you a single string, this one doesn't. I'd actually be inclined to switch around this one and the next one...
A complication I just thought of is that you can have chained assignment within a sequence unpacking assignment:
spam, eggs, aardvark = foo = bar, hovercraft = 'abcd'
and the other way around:
spam = eggs = (aardvark, foo, bar) = hovercraft = 'abc'
That's going to make things tricky.
Very very good point. Ouch. It'd probably be safest to define it to always be a single string, and then have a fully-nestable and recursive system. So instead of simply separating with comma or equals or whatever, it might be best to group AND separate. spam, eggs = "@@" # "[spam,eggs]" spam = eggs = "@@" # "{spam,eggs}" (spam, eggs) = (foo, bar) = "@@" # "{[spam,eggs],[foo,bar]}"
Questions:
1) Is this restricted to the "=" assignment operator, or will other operators trigger this too? x += f(@@) # ? if x := f(@@): # ?
I hadn't thought that far ahead. I did think that we ought to exclude the walrus operator because it would be ambiguous:
spam = eggs * (cheese:=foo+bar(@@))
Does @@ get the value 'cheese' or 'spam'? If we require an assignment statement, then it can only be 'spam' and the ambiguity is gone.
Yeah, I'd agree about :=. Less clear about +=, but as with some of the others, it may be best to (a) leave it restricted with room for expansion, and/or (b) let a reference implementation guide the decision.
2) What about other forms of assignment? for spam in foo(@@): # ?
YAGNI.
Absolutely agree, especially because of this case: spam = [ord(x) for x in @@] If for loops on their own don't define an @@ target, then for loops inside comprehensions won't either, and this wouldn't be ambiguous.
3) Is this a string literal, or a magic token that happens to evaluate as a string?
An actual string.
Either way, it would be a string. The difference is that string literals can be placed adjacent to each other:
"{1}" f' - {1+2=} - ' '{2}' '{1} - 1+2=3 - {2}'
Which goes to show, btw, that an f-string is still a literal, even though it's not a constant.
x = @@ ".json" # Legal if @@ is a string literal
Heh, I wouldn't necessarily require that. (Nor would I object to it.) Implicit string concatenation is a nice feature, but I'm not sure we want to extend it. Its not hard to slot an explicit `+` in there.
True. Probably another thing best guided by the reference implementation. I think all these open questions are minor details, but the core proposal is strong enough to handle the uncertainty. Might be worth starting a dedicated thread for it. ChrisA
On 5/26/2021 7:07 AM, Chris Angelico wrote:
Either way, it would be a string. The difference is that string literals can be placed adjacent to each other:
"{1}" f' - {1+2=} - ' '{2}' '{1} - 1+2=3 - {2}'
Which goes to show, btw, that an f-string is still a literal, even though it's not a constant.
Again unrelated to the topic at hand, but I think it's interesting to see what's going on behind the scenes:
dis.dis("'{1}' f' - {1+2=} - ' '{2}'") 1 0 LOAD_CONST 0 ('{1} - 1+2=') 2 LOAD_CONST 1 (3) 4 FORMAT_VALUE 2 (repr) 6 LOAD_CONST 2 (' - {2}') 8 BUILD_STRING 3 10 RETURN_VALUE
The 1+2 expression is replaced by 3 by some optimizer step. Regular strings and the literal part of f-strings are merged by the f-string "compiler". I keep forgetting about this behavior. I usually start doubling the braces, but this auto-concatenation is probably a better idea. Eric
On Tue, May 25, 2021 at 12:30 AM Steven D'Aprano <steve@pearwood.info> wrote:
Your proposal appears to be:
@decorator(expression) targetname
# transformed into:
targetname = decorator("targetname", expression)
Correct
But in class and function decorator contexts, the equivalent syntax is equivalent to:
@decorator("spam this") def func(): pass
# transformed to:
def func(): pass func = decorator("spam this")(func)
except that in CPython it is not actually implemented as two separate steps like that.
But the critical difference is that the argument "spam this" should be passed to the decorator *factory*, which then returns the actual decorator that gets applied to the variable. (Or function/ class in the case of regulator decorator syntax.)
Yes, you are correct that the current only use of a decorator, if it takes arguments, first executes the decorator factory and then passes in the function object. This proposal is more restrictive, and always calls the decorator once, whether parentheses follow the decorator name or not. I suppose the same could be done for variable decorating. It would look something like: @namedtuple("x y z") Point would translate to Point = namedtuple("x y z")("Point") only namedtuple doesn't work that way, so a new decorator factory would have to be created @namedtuple_declare("x y z") Point where namedtuple_declare worked something like this? @classmethod def namedtuple_declare(field_names, *, rename=False, defaults=None, module=None): def delayed_namedtuple(typename): return namedtuple(typename, field_names, rename=rename, defaults=defaults, module=module) return delayed_namedtuple Which maybe would turn out to be useful as you could pre-bake the decorator function. Others may feel differently but I don't find that as compelling as the original restricted proposal. # now possible, but what does this get you over different instances of the same # class? point_tuple = namedtuple_declare("x y z") @point_tuple Origin @point_tuple Inflection @point_tuple Point Of course even following the function decorators, the most basic uses would still act the same. @str GREEN @symbols x However, a lot of the "for free" uses I outlined would no longer be possible, and new factory functions would have to be added to the standard lib to facilitate creating a variable decorator for these use cases. I realize the original proposal acts differently than current decorators, it's not currently valid to have the @ on the same line as the def, so until now it was inherently a multi-line statement but I was proposing a one-line only statement. The use of @ is not necessary to the overall proposal. I chose it because current users of python would understand that somewhere there is a function of the same name that is "getting in the way" of the binding. Also it visually draws attention to itself when scanning down the left hand side unlike some other options such as "as" which has historically been the only use of bind to the right.
To my mind, the only interesting part of this proposal is access to the binding target name. If we remove that from the proposal, variable decorators are nothing more than function calls, and we already can do that:
variable = decorator("spam this")
Yes, this was the only reason I had to bring this proposal forward. Although it could be argued that function decorators are still only function calls, they have proven to be popular and IMO create cleaner reading code.
Here's a counter-proposal: we have a special symbol which is transformed at compile-time to the left hand assignment target as a string. Let's say we make that special expression `@@` or the googly-eyes symbol.
Very interesting idea. I will say that turning this proposal back into something to be used in assignment does expand it reach, by how much would have to be decided, and can make for more confusion. You bring up spam = eggs = cheese = func(arg, @@) but what about spam = (eggs := func(@@, flags)) I think I know what the @@ is supposed to turn into. At least I do if @@ is valid in assignment expressions. If it is not, did I remember that it always binds to the assignment statement name? I will say that this offers more flexibility in that the binding name can be reused anywhere on the lhs, although I don't think it's common to have these sorts of factories where the first argument isn't the name. I think it does place a larger cognitive burden on the developer (very small, but still larger IMO). They will still have to remember what the symbol is for name replacement and be willing to type that in instead. Will python users reach for this in the simple case? RED = "RED" is maybe faster to get out of my fingers than pausing and writing RED = @@. Will some scripts start to look like this? PATH = os.getenv("PATH") VERY_LONG_USER_OPTION_TO_TYPE = os.getenv(@@) OTHEROPTIONHARDTOREAD = os.getenv(@@) PYTHONHOME = os.getenv("PYTHONHONE") # oops! This is maybe overly critical, and I realize there is nothing stopping decorators from ending up just like this, but I think that if a developer starts out writing an assignment statement they might just finish it without reaching for @@. Starting a new line with a decorator is a distinct way of binding names. Some things would be more intuitive though, especially around dicts and general uses of square brackets. header = message.get(@@) @message.get header verbose = @@ in options @operator.contains(options) verbose RED = @@
# transforms to `RED = 'RED'` ... If there's no target, it resolves to None: ... Chained assignments transform to a tuple of target names: ... Sequence unpacking assignment gets transformed as a single comma-seperated string:
This feels like a lot of rules to keep straight, and a lot of verbose checking required for factory functions that want to support all three (four?! I would also be in favor of SyntaxError) possibilities. Target resolution is performed at compile-time, not runtime. There's no
global variable called "@@". That means that this won't work:
code = compile("type(@@, bases, namespaces)", '', 'single') # above transforms to `type(None, bases, namespace)` myclass = eval(code)
But I think that restriction is fine.
Agreed A very interesting counter. Lots to consider here, appreciate the detailed reply! Regards ~Jeremiah
I had a similar idea ~8 years ago while working on a RAD (rapid application development) framework [*] that had to manage business objects using a variety of frameworks: an ORM (SQLAlchemy), a full-text engine (Whoosh), as well as a specifically developed CRUD Web UI framework, permission system, audit system, etc. In this context, variable decorators, in addition to type annotations, could bring a whole new level of internal DSL expressivity, including for instance: - Annotation to express data access and structural constraints (support for DBC e.g. "@constraints" or alternative constructs for some Attrs / Dataclass features) - Annotations to express ORM features (à la Java's JPA): @Id, @NotNull, @OneToOne, @ManyToMany, etc. - Annotations to express serialization (think Marshmallow or Pydantic) - Annotation to express UI hints (e.g.: @widget(type=RichText, size="300px", max_length=500, color=GREEN, required=True, ...)) - Annotations to express full-text search indexability (e.g. @indexed) - Annotation to express field-level permissions: @acl(...) or @permission(...) etc. This could provide more elegant, and better decoupled, syntax for things that are currently done using either metaclasses (e.g. Django ORM or SQLAlchemy ORM) or the "class Meta" idiom used by frameworks such as Django or Marshmallow. S. [*]: since this was not available, we eventually went with an external DSL in combination with the "info" argument to the Column declaration in SQLAlchemy, used to pass metadata useful for our framework. This worked, but the code could have been much more elegant with variable decorators (in other words, it was quite ugly in some places and harder than necessary to maintain). On Tue, May 25, 2021 at 3:39 AM micro codery <ucodery@gmail.com> wrote:
Variable decorators have been suggested here before, as have new statements that could also achieve the same level of access to the binding name. However I propose a much more restricted syntax that would make for less edge cases where what is actually passed to the decorator callable may be ambiguous.
Basically this would add syntax to python that would transform @decorator("spam this") variable into variable = decorator("variable", "spam this")
The decorator would be limited to being on the same line as the variable name. Additionally, the variable can not also be assigned to on the same line, so @decorator("spam this") variable = eggs would be a SyntaxError. Annotating these variable would be allowed @decoratopr("spam this") variable: Food The actual decorator would follow the same rules as current decorators in that the parentheses can be omitted if no other arguments are required to the function and any arbitrary expression can be used, however the same restraint should be cautioned as is for current decorators post PEP 614.
The most straightforward use of this decorating would be that str() could be used without change to declare variables that are strings of the same name, generally cutting down on repeating yourself when declaring constants.
#instead of GREEN = "GREEN" @str GREEN
The classic example of a factory that needs its target name as a string value is namedtuple. I don't think this example is enough alone to sway this list but I bring it up as it has also been at the front of past ideas to allow variable decorating.
#instead of Point = namedtuple("Point", "x y z") @namedtuple("x y z") Point
I have sometimes heard that dataclasses make this no longer an issue, implying that dataclasses can be used in place of namedtuple in any new code. Putting aside that dataclasses are not a strict replacement for namedtuple, the dataclasses module continues this factory design with the make_dataclass function.
#instead of Point = make_dataclass("Point", [("x", int), ("y", int), ("z", int)]) @make_dataclass([("x", int), ("y", int), ("z", int)]) Point
Enum is one more case where there is a factory equivalent to the class declaration.
#instead of Colors = Enum("Colors", "RED GREEN BLUE") @Enum("RED GREEN BLUE") Colors
If you want each enum member value to match its name there is even more repetition, however by using the original example for decorated variables, this can be reduced when using the more common class style.
class Colors(Enum): @str RED @str GREEN @str BLUE
One final standard library module that I will mention because it seems to keep adding similar factory functions is typing. It already has the factory functions NewType, TypeVar, ParamSpec, TypedDict, NamedTuple. Their use with the new variable decorator would look much the same as earlier examples, but here would be one case where type hinting the variable in such a statement could aid the decorator function
#instead of UUIDType = NewType("UUIDType", str) @NewType UUIDType: str
If adopted, no change to these functions would be necessary to allow their use as a variable decorators as they all already take the name as a string in the first argument. Additionally, as variable decorators are actually callables just like the current decorators, no new restrictions are being forced on users of the language. If a user really wants to create a namedtuple with a typename of Foo but assign the result to a variable named ___trippleS33cret___name__here_ the language will not stop them.
I would very much like to know if any module maintainers are interested in this syntax. Convincing arguments from the standard lib (whether you consider the above convincing or not) are probably not enough to get a syntax change into python. But if enough third party modules want to use this functionality I believe that would be the real push to get it done.
Specifically this idea first started growing when I read on another ideas message that using the module sympy would often start with a line like x = symbol('x') which, with no change, could be used as a variable decorator @symbol x Now what may be even more typical (I really have never used the module myself) is defining all needed symbols on one line with x, y, z = symbol('x y z') which could conceivably be rewritten like @symbol x, y, z However I have not yet been able to decide if multi-assignment would be overall beneficial to this new syntax. Probably not, at least at this stage. It would not be hard for a decorator function to return multiple values, but how would the multiple names be passed in? A string with whitespace like this original, and namedtuple's field_names, is possible as no valid identifier can have a space in it. However, it forces any would-be variable decorator to know this and check if a space is in its first argument, even if it cannot produce multiple values. Another option is to pass in an iterable, but the same additional effort would be placed on decorators that otherwise don't want to deal with multi-assignment. Unpacking the names as the first n arguments would mean that decorators that just want one variable can get just that and those that may want more will also get them, but then practically no variable decorator could take additional non-keyword arguments.
Regards ~ Jeremiah _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/2FOIN5... Code of Conduct: http://python.org/psf/codeofconduct/
-- Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - http://linkedin.com/in/sfermigier Founder & CEO, Abilian - Enterprise Social Software - http://www.abilian.com/ Chairman, National Council for Free & Open Source Software (CNLL) - http://cnll.fr/ Founder & Organiser, PyParis & PyData Paris - http://pyparis.org/ & http://pydata.fr/
On Tue, May 25, 2021 at 10:55:07AM +0200, Stéfane Fermigier wrote:
I had a similar idea ~8 years ago while working on a RAD (rapid application development) framework [*] that had to manage business objects using a variety of frameworks: an ORM (SQLAlchemy), a full-text engine (Whoosh), as well as a specifically developed CRUD Web UI framework, permission system, audit system, etc.
In this context, variable decorators, in addition to type annotations, could bring a whole new level of internal DSL expressivity, including for instance:
Could you explain exactly how this proposed feature would allow us to do all the things you say it would allow us to do? It's not clear to me how knowing the binding name on the left would allow us to "express data access constraints" etc. E.g. you say:
- Annotations to express full-text search indexability (e.g. @indexed)
I don't even know how to begin interpreting how you get there from the proposed syntax: @function(args) name # -> `name = function('name', args)` [...]
[*]: since this was not available, we eventually went with an external DSL in combination with the "info" argument to the Column declaration in SQLAlchemy, used to pass metadata useful for our framework. This worked, but the code could have been much more elegant with variable decorators (in other words, it was quite ugly in some places and harder than necessary to maintain).
The only think this proposal adds that isn't available right now with a regular function call is that it automagically inserts the assignment target name into the function call arguments. I'm honestly not seeing how you go from such a niche piece of functionality to your glowing review of how it would make all these things so much more elegant. Can you give concrete examples please? As I see it, objects should almost never know or care what names (note plural) they are known by, or even if they are bound to any names at all. There are a few exceptions, mostly classes and sympy symbols, but I expect that this functionality should be rather niche. -- Steve
On Tue, May 25, 2021 at 12:14 PM Steven D'Aprano <steve@pearwood.info> wrote:
On Tue, May 25, 2021 at 10:55:07AM +0200, Stéfane Fermigier wrote:
I had a similar idea ~8 years ago while working on a RAD (rapid application development) framework [*] that had to manage business objects using a variety of frameworks: an ORM (SQLAlchemy), a full-text engine (Whoosh), as well as a specifically developed CRUD Web UI framework, permission system, audit system, etc.
In this context, variable decorators, in addition to type annotations, could bring a whole new level of internal DSL expressivity, including for instance:
Could you explain exactly how this proposed feature would allow us to do all the things you say it would allow us to do? It's not clear to me how knowing the binding name on the left would allow us to "express data access constraints" etc.
I was thinking at the framework level, e.g. the decorator would be use to register some metadata somewhere (either class-level or in a global registry, say), and the framework would be to check permission against the current user when accessing the object's field (e.g. at serialisation / deserialisation time, and/or when rendering templates). For all the things I have suggested, what I have in mind is just: 1) Register some metadata at decorator execution time 2) The framework uses these metadata when it needs them
E.g. you say:
- Annotations to express full-text search indexability (e.g. @indexed)
I don't even know how to begin interpreting how you get there from the proposed syntax:
@function(args) name # -> `name = function('name', args)`
For the use cases I have in mind, some kind of context information would need to be passed too, like: class Customer: @permissions({"read": USER, "write": MANAGER}) first_name: str the function primary_key should be called with the class being constructed for it to make sense. Here are some alternatives: 1) Set metadata directly on some magical attribute: class Customer: __permissions__ = { 'first_name': { READ: ..., WRITE: ... } } first_name: str cons: risk of mispelling or forgetting some attributes; lack of locality; how do we deal with inheritance ? 1bis): class Customer: class Meta: permissions__ = { 'first_name': { READ: ..., WRITE: ... } } first_name: str This is similar to the previous one, so same issues. 2) Dexterity syntax (I'm not a user of Dexterity, so I might be wrong, this is from: https://www.stevemcmahon.com/classes/dexterity/dexterity%20class%202013.pdf ): class Customer: dexterity.read_permission( first_name='cmf.ReviewPortalContent' ) dexterity.write_permission( first_name='cmf.ReviewPortalContent' ) first_name: str con: easy to misspell the field name (hopefully the rest of the issues is taken care by the framework) 3) One possible idiom to use decorators as I think would be fit, without changing the language, but at the price of an extra line and some additional mental burden: class Customer: first_name: str @acl({"read": USER, "write": MANAGER}) def meta_first_name(self): pass I.e. use a method decorator on a dummy function named similarly to the variable (e.g. by prepending some magical prefix). The obvious con is that we still haven't prevented the risk of typo in the function name. [...] As I see it, objects should almost never know or care what names
(note plural) they are known by, or even if they are bound to any names at all. There are a few exceptions, mostly classes and sympy symbols, but I expect that this functionality should be rather niche.
For the kind of applications I have been doing over the last 20 years (enterprise content and information management applications), this is not niche, this is where 50% of the work goes (i.e. defining object models, permissions, workflows, forms, validation, etc.). S. -- Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - http://linkedin.com/in/sfermigier Founder & CEO, Abilian - Enterprise Social Software - http://www.abilian.com/ Chairman, National Council for Free & Open Source Software (CNLL) - http://cnll.fr/ Founder & Organiser, PyParis & PyData Paris - http://pyparis.org/ & http://pydata.fr/
On Wed, May 26, 2021 at 08:58:51AM +0200, Stéfane Fermigier wrote:
For the use cases I have in mind, some kind of context information would need to be passed too, like:
class Customer:
@permissions({"read": USER, "write": MANAGER}) first_name: str
the function primary_key should be called with the class being constructed for it to make sense.
What function primary_key? Never mind, it probably doesn't matter. According to this proposal, that example would be illegal. You have a decorator with no variable. You could write this instead: @permissions({"read": USER, "write": MANAGER}) first_name: str which would be transformed to: first_name:str = permissions('first_name', {"read": USER, "write": MANAGER}) I think that's a minor win in that you don't have to repeat the variable name. Using my counter-proposal for a googly-eyes symbol: first_name:str = permissions(@@, {"read": USER, "write": MANAGER}) I think that's better as it is explicit that an assignment is happening.
Here are some alternatives:
1) Set metadata directly on some magical attribute:
class Customer: __permissions__ = { 'first_name': { READ: ..., WRITE: ... } }
first_name: str
cons: risk of mispelling or forgetting some attributes; lack of locality; how do we deal with inheritance ?
The inheritance question applies equally to the decorator version. [...]
class Customer: first_name: str
@acl({"read": USER, "write": MANAGER}) def meta_first_name(self): pass
I.e. use a method decorator on a dummy function named similarly to the variable (e.g. by prepending some magical prefix).
Good lord. All that trouble to avoid typing the name of the variable as a function argument, and then you still end up typing it in the dummy function name :-( first_name = acl('first_name', {"read": USER, "write": MANAGER}) is *much* clearer and simpler than any of the hacks you have shown, especially the one with the dummy method. Seriously, you end up typing an extra 21 characters plus newlines and indents to avoid typing 'first_name' as a function argument. That's just sad. Thank you for your examples. They suggest to me that: - There are more uses for knowing the assignment target than I expected. - Frameworks go through a huge amount of (unnecessary?) work to avoid writing the name of a varible -- and then end up writing it anyway. - Decorators are a hammer, and some people think that every problem is a nail. - There is good case for having a feature that gives the right-hand side of assignment statements access to the assignment targets as strings. - But I still don't think that decorator syntax is the right solution. -- Steve
On Wed, May 26, 2021 at 08:10:17PM +1000, Steven D'Aprano wrote:
- Decorators are a hammer, and some people think that every problem is a nail.
Sorry, on re-reading that statement, it comes across as less gracious than I intended. So let me explicitly thank Jeremiah for raising this issue, hammer or not :-) -- Steve
On Wed, May 26, 2021 at 12:13 PM Steven D'Aprano <steve@pearwood.info> wrote:
On Wed, May 26, 2021 at 08:58:51AM +0200, Stéfane Fermigier wrote:
For the use cases I have in mind, some kind of context information would need to be passed too, like:
class Customer:
@permissions({"read": USER, "write": MANAGER}) first_name: str
the function primary_key should be called with the class being constructed for it to make sense.
What function primary_key? Never mind, it probably doesn't matter.
Sorry, I re-wrote my example after a first draft. In this example "primary_key" should be "permissions". I think that's a minor win in that you don't have to repeat the variable
name.
Repetition is not the main issue, in my own experience. The main issue is the risk of making a typo in the variable's name when passing it as a string (it happens to me all the time, specially when copy-pasting code and forgetting to change the variable's name in every place).
1) Set metadata directly on some magical attribute:
class Customer: __permissions__ = { 'first_name': { READ: ..., WRITE: ... } }
first_name: str
cons: risk of mispelling or forgetting some attributes; lack of locality; how do we deal with inheritance ?
The inheritance question applies equally to the decorator version.
What I meant is that if you set up the metadata registry by yourself, that won't be enough to deal with inheritance. If the decorator takes care of it, it can probably be made smart enough to massage the provided metadata into the information needed at runtime. But I agree that's mostly irrelevant for the discussion at hand.
class Customer: first_name: str
@acl({"read": USER, "write": MANAGER}) def meta_first_name(self): pass
I.e. use a method decorator on a dummy function named similarly to the variable (e.g. by prepending some magical prefix).
Good lord. All that trouble to avoid typing the name of the variable as a function argument, and then you still end up typing it in the dummy function name :-(
first_name = acl('first_name', {"read": USER, "write": MANAGER})
is *much* clearer and simpler than any of the hacks you have shown, especially the one with the dummy method. Seriously, you end up typing an extra 21 characters plus newlines and indents to avoid typing 'first_name' as a function argument. That's just sad.
Well that was just a straw man just to convey what I'd like to achieve, i.e. a variable decorator should be, mostly equivalent to this semantically. I've never implemented this idea. Thank you for your examples. They suggest to me that:
[...]
- Decorators are a hammer, and some people think that every problem is a nail.
Actually there is a bit more to it. If we go back to the name, "decorators" are supposed to "decorate" things (i.e. add metadata). That's what they do, and nothing more, in Java, for instance. (I know that in Python they do more.) Now I want to add metadata to some variables, that's quite logical that I would want to use some kind of "decorator" for that. Another thing is that there are decorators for classes, methods, functions. My humble opinion is that they would be also useful for class variables (and probably for other variables too). Last point that I like in the decorator syntax: it's I can compose N different decorators and keep the intent obvious: @acl(READ, WRITE) @constraint(10 < _ < 100) @not_null @indexed @depends_on(whatever) @inject @... first_name: str (Yes, it can become a bit heavy after some point, but imagine that all these metadata are mandated by the business domain, what would be the alternative syntaxes or approaches ?). - There is good case for having a feature that gives the right-hand
side of assignment statements access to the assignment targets as strings.
Yep.
- But I still don't think that decorator syntax is the right solution.
Not usually fond of adding new stuff to the language, but as I wrote, this is something that I was ready to argue about 8 years ago, I never did because I was in a hurry delivering the features needed for my customers, but I'm still fond of the idea of moving in this direction. S. -- Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - http://linkedin.com/in/sfermigier Founder & CEO, Abilian - Enterprise Social Software - http://www.abilian.com/ Chairman, National Council for Free & Open Source Software (CNLL) - http://cnll.fr/ Founder & Organiser, PyParis & PyData Paris - http://pyparis.org/ & http://pydata.fr/
On Wed, May 26, 2021 at 01:33:07PM +0200, Stéfane Fermigier wrote:
I think that's a minor win in that you don't have to repeat the variable name.
Repetition is not the main issue, in my own experience. The main issue is the risk of making a typo in the variable's name when passing it as a string (it happens to me all the time, specially when copy-pasting code and forgetting to change the variable's name in every place).
Right, that's the problem with having to repeat the name. The issue is minor when you only have to do it once or twice, especially if the name is only informative rather than functional: Spam = namedtuple('Spam', *fieldnames) Eggs = namedtuple('Egs', *otherfields) # Oops The second line, with it's typo, doesn't really matter. The class name is just informative, it's not functional. But in your framework examples, it does matter: personal_name = permissions('personnal_name', ...) # Oops family_name = permissions('family_name', ...) address = permissions('family_name', ...) # copy-paste error # and many more examples The repetition is an opportunity to slip in functional bugs, not just trivial typos.
Last point that I like in the decorator syntax: it's
I can compose N different decorators and keep the intent obvious:
@acl(READ, WRITE) @constraint(10 < _ < 100) @not_null @indexed @depends_on(whatever) @inject @... first_name: str
Hmm, that's a good point. On the other hand, it would be terribly confusing if the same syntax: @decorate had radically different meaning depending on whether it was followed by a class/function or a bare name. -- Steve
participants (8)
-
Chris Angelico -
Eric V. Smith -
Ethan Furman -
micro codery -
Ricky Teachey -
Stephen J. Turnbull -
Steven D'Aprano -
Stéfane Fermigier