Re: [Python-ideas] Decorators for variables
(Resending to correct list. Sorry about that.) On Fri, Apr 1, 2016 at 11:25 AM, Ian Kelly <ian.g.kelly@gmail.com> wrote:
Le 01/04/2016 19:33, Ian Kelly a écrit :
Not saying I like the proposal, but you can argue against regular decorators the same way: @foo def bar(): pass Is just: bar = foo(bar) But, I think the benefit for @decorator on functions is mainly because a function body is big, and this way we can read the decorator next to the function signature while on a variable, this just add another way to call a function on a variable.
On Sat, Apr 2, 2016 at 4:53 AM, Matthias welp <boekewurm@gmail.com> wrote:
Wait, what? Function decorators are simply higher-order functions: they take a function as an argument, and return a function [1]. They can't change the behaviour of the name, only the value it's bound to. ChrisA [1] Usually. Nothing's stopping them from returning non-callables, except that it'd confuse the living daylights out of people.
Function decorators
There are decorators that return a callable that not calls the function that was given as an argument, but also do some other things, and therefore change the behaviour of that function.
If possible, yes. So that there is a standardized way to access changing variables, or to put limits on the content of the variable, similar to the @accepts and @produces decorators that are seen here ( https://wiki.python.org/moin/PythonDecoratorLibrary#Type_Enforcement_.28acce... )
On 04/01/2016 11:08 AM, Matthias welp wrote:
Even earlier, Ethan Furman wrote:
I don't see it happening. Making that change would be a lot of work, and the advantages (if any) of the second method over the first do not warrant it. -- ~Ethan~
On Fri, Apr 1, 2016 at 2:09 PM Matthias welp <boekewurm@gmail.com> wrote:
There is a standardized way. You can extend ``property`` or mimic its implementation. Then instead of class Foo: a = property(getter, no_cycles) You can write class Foo: a = NoCycles() I haven't linked to a how-to for doing this is, because I think it's unnecessary for most small projects. Every so often someone asks for a more pleasant syntax for specifying a property, getter and setter. Guido seems to consistently reply that he thinks our current situation is good enough. I'd dig up a link to the email archive for you, but Google wasn't being very kind to me.
On Fri, Apr 01, 2016 at 08:08:54PM +0200, Matthias welp wrote:
But decoratoring a *name binding* isn't going to do that. All it will do is limit the *initial value*, exactly as the function call does. After calling a = Char(length=10, value='empty') the name "a" is bound to the result of Char(...), whatever that happens to call. But that doesn't change the behaviour of the *name* "a", it only sets the value it is bound to. Nothing stops anyone from saying: a = 42 and re-binding the name to another value which is no longer a Char. Python has no default support for running custom code when binding arbitrary names to a value. To get this sort of thing to work, you are limited to attributes, using the descriptor protocol. I'm not going to explain descriptors now, you can google them, but property is a descriptor. So let's imagine that Python allows the @ syntax as you request, and go through the cases to see what that would imply. For local variables inside functions, or global top-level module variables, it doesn't give you any interesting power at all. All you have is an alternative syntax: @spam @eggs @cheese x = 999 is exactly the same as x = spam(eggs(cheese(999))) right now. You don't even save any characters: 28 (including newlines) for both. But this doesn't give us anything new and exciting, since x is now just a regular variable that can be replaced with some other value. So in this scenario, this sounds boring -- it gives you nothing you don't already have. Inside a class, we have the power of descriptors available to us, so we can use them for computed attributes. (That's how property works, among many others.) So suppose we have: class MyClass(object): @decorate x = 999 instance = MyClass() Now instance.x can be a descriptor, which means that it can enforce type and value validation rules, or logging, or whatever amazing functionality you want to add. This is good. But you can already do this. You just have to write: class MyClass(object): x = decorate(999) instead. If decorate() returns a descriptor, it returns a descriptor whatever syntax you use. What benefit do you gain? Well, in the function and class decorator case, you gain the benefit that the decoration is close to the class or function header, and you don't have to write the name three times: class MyClass(object): def method(self): [... many, many, many lines of code ...] method = decorate(method) But this isn't an advantage in the case of the variable: class MyClass(object): x = decorate(value) You only have to write the name once, and the decoration is not going to be far away. So just like the global variable case, there's no advantage to @decorate syntax, regardless of whether you are in a class or not. -- Steve
This is exactly my point. When they implemented descriptors (2.2) they did that to add to the new-style class system. I think they did a great job, but namespaces were overlooked in my opinion. Why should we be limited to class attributes when using descriptors? Any namespace which contains a '.__dict__' content mapping should be able to hold descriptors in my opinion. If this has been discussed before, then please link me to the relevant discussion, I'd love to read the points made.
But this isn't an advantage in the case of the variable
Assigning variables from function results is fairly common. When chaining function calls to get the desired behaviour of the variable, it will get confusing which part is the 'value' part, and which part is the 'behaviour' part, apart from namings: var = logging( require(int)( factorize( 4 )))) would get more clear if you wrote it this way: @logging @require(int) var = factorize(4) Something I just thought about: currently you can use the property() call to make an attribute have descriptor properties. This may be somewhat controversial, but maybe limit this to decorators only? While keeping the '.__dict__' override method open, the way things work currently won't change that much, but it will make the assignment of attributes or variables with descriptor properties a lot more intuitive, as you do not set a *name* to a variable and then can undo that just a while later: var = property(setter, getter, deleter, docs) var = 20 currently changes behaviour depending on what kind of scope it is located in (class description, any other scope), while decorators (for functions at least) work in every scope I can think of. I think that is strange, and that it should just be the same everywhere. Using decorators here could be a very nice and interesting choice.
Sorry, that was untested code. My expectations of class definitions was wrong, as it does not actually change behaviour inside it's own scope. I thought that when you are defining a class, that when you assign a property value to an attribute, that the attribute 'name value' will directly change it's behaviour to include the descriptor properties of the property object assigned. My mistake. On 3 April 2016 at 08:32, Chris Angelico <rosuav@gmail.com> wrote:
On Sun, Apr 3, 2016 at 4:53 PM, Matthias welp <boekewurm@gmail.com> wrote:
Ah. There is a significant difference between assignment within a class definition and assignment from a function _inside_ that class definition, but in any given scope, double assignment always does the same thing: last one wins. Which is a good thing, when it comes to the @property decorator: class LifeAndUniverse: @property def answer(self): return 42 @answer.setter def answer(self, value): print("No fair changing the answer!") @answer.deleter def answer(self): print("You just deleted.... everything.") Each function definition overwrites the previous "answer" with a new one, which (thanks to the way setter and deleter are implemented) incorporates the previous code, but nothing in Python mandates that. So is there anything left of the assignment-decorator proposal, or is it completely withdrawn? (I always like to read over even the bad proposals - there's often something good in them, Martin Farquhar Tupper's "Proverbial Philosophy" aside.) ChrisA
So is there anything left of the assignment-decorator proposal, or is it completely withdrawn?
I think this sums the current open ends up: - Namespace variables decoration was dismissed by one of Steven's posts, but is actually something that might be wanted (via being able to put descriptors into namespaces that have a __dict__ accessor (e.g. modules)) - Variable decoration can be more clear about descriptor/value difference at assignment - Giving property objects access to their variable's name (e.g. via __name__) like functions have would open up quite a bit of possibilities, and would mean decorators would get quite a bit more power than what they have. Something that I had said earlier, but what went on a sidepath - Decorators may directly *deep* set the behaviour of the variable, and with it set the further behaviour of the variable (in the same scope). Such that @decorator var = 20 var = 40 will not reset var to 40, but the var = 40 goes through the descriptor (if applied).
On Sun, Apr 3, 2016 at 6:02 PM, Matthias welp <boekewurm@gmail.com> wrote:
None of this is possible with decorators *per se*, as they simply mutate an object as it goes past. The magic you're seeing (and yearing for) comes from two places: functions have names (so the bit in "def SOMETHING(...)" actually affects the object constructed, not just the name it's bound to), and attribute access and the descriptor protocol. The latter is what makes @property work, but you don't need decorators to use descriptor protocol - in fact, it's crucial to the way bound methods are created. So, let's take your proposals one by one.
You'll need to elaborate on exactly what this can and can't do.
- Variable decoration can be more clear about descriptor/value difference at assignment
All Python names are in namespaces, so I'm not sure how this is different from the above.
This is fundamentally not possible in the general case; however, you could easily make a special form of the @property decorator which *requires* that the attribute name be the same as the function name that implements it, and then it would work (because function names are attributes of function objects, while "the name this is bound to" is not an attribute of anything). It's already accessible, as ClassName.property_name.fget.__name__; you could make this more accessible or more discoverable by subclassing property. In fact, you can even go meta:
However, this is vulnerable to the same thing that any other "what is my name" system has: you can rebind anything.
So there's fundamentally no way to say "what name is bound to me"; but you can quite effectively say "what is my canonical name", depending on a function to provide that name. ChrisA
On Sun, Apr 03, 2016 at 10:02:55AM +0200, Matthias welp wrote:
It's not that I dismissed the idea, but that such a thing is not possible today. Maybe you could hack up something tricky by replacing the module __dict__ with a subclass, or replacing the entire module object with some custom class instance, but that's messy and fragile, and I'm not sure that it would work for (say) globals inside functions. I think the idea of "namespace descriptors" is promising. But it probably needs a PEP and a significant amount of work to determine what actually is possible now and what needs support from the compiler.
- Variable decoration can be more clear about descriptor/value difference at assignment
I'm not sure what you mean by that.
- Giving property objects access to their variable's name (e.g. via __name__) like functions have would open up quite a bit of possibilities,
Such as what?
and would mean decorators would get quite a bit more power than what they have.
What extra power are you thinking of?
This functionality has nothing to do with decorators. If it were possible, if namespaces (other than classes) supported descriptors, then the decorator syntax doesn't add anything to this. You could write: var = magic(20) I'm not calling it "decorator" because the whole @ decorator syntax is irrelevant to this. With respect Matthias, I think you are conflating and mixing up the power of descriptors as applied to classes, and the separate and distinct powers of decorators. You don't need one to have the other. -- Steve
On Saturday, April 02, 2016 23:33, Chris Angelico wrote:
I can sort of see it, like this? class C(): def __init__(self): self.x = 99 @property def f(self): return self.x x = 101 @property def f(namespace): return namespace.x c = C() print(c.x) 99 print(c.f) 99 print(C.f.fget(c)) 99 print(x) 101 Here's the inconsistency, should implicitly bind to a "global namespace object". print(f) <property object at 0x00000000023FB6D8> In other words, something like this: class GNS: def __getattr__(self, name): prop = globals().get(name) if isinstance(prop, property): return prop.fget(self) return prop global_namespace = GNS() print(x is global_namespace.x, "should be 'True'") True should be 'True' print(global_namespace.f) 101
On Fri, Apr 01, 2016 at 07:44:21PM +0200, Michel Desmoulin wrote:
Not quite. It's actually: def bar(): pass bar = foo(bar) which is *three* uses of the same name, two of which may be a long way from the first. As you say:
But, I think the benefit for @decorator on functions is mainly because a function body is big,
That is certainly part of the justification for @ syntax. It might help to remember that decorators themselves have been possible in Python going all the way back to Python 1.5, at least, if not earlier, and the first three decorators in the standard library (property, classmethod and staticmethod) were added a few releases before decorator syntax using @. (If I remember correctly, property etc were added in 2.2 but @ syntax wasn't added until 2.4.) So the decorator concept had many years to prove itself before being given special syntax. The two reasons for adding extra syntax were: (1) to keep the decoration near the function declaration; and (2) to avoid having to repeat the function name three times.
I'm afraid I don't understand what you mean by this. What does "while on a variable" mean here? -- Steve
Steven D'Aprano writes:
Missing punctuation (see substring in square brackets): and this way we can read the decorator next to the function signature[. W]hile on a variable, this just add another way to call a function on a variable. So, it appears that he's proposing pure syntactic sugar with no semantic change at all. The syntactic effects are that decorator syntax separates function calls from the values they act on (-1 on that), and makes it appear that in Python a function can change the value of its formal argument (-1 on that, too). The latter characteristic may need some expansion. Of course Python has mutable objects which can be changed "inside" a function and that effect propagates to "outside" the function. But in the containing scope, while the object has mutated, the variable (more precisely, the name-object binding) has not. It seems to me that everything he's asked for can be accomplished by defining a descriptor class and assigning an instance of that class. I don't yet understand why that isn't good enough, unless it's just a matter of taste about the sweetness of syntax.
Le 01/04/2016 19:33, Ian Kelly a écrit :
Not saying I like the proposal, but you can argue against regular decorators the same way: @foo def bar(): pass Is just: bar = foo(bar) But, I think the benefit for @decorator on functions is mainly because a function body is big, and this way we can read the decorator next to the function signature while on a variable, this just add another way to call a function on a variable.
On Sat, Apr 2, 2016 at 4:53 AM, Matthias welp <boekewurm@gmail.com> wrote:
Wait, what? Function decorators are simply higher-order functions: they take a function as an argument, and return a function [1]. They can't change the behaviour of the name, only the value it's bound to. ChrisA [1] Usually. Nothing's stopping them from returning non-callables, except that it'd confuse the living daylights out of people.
Function decorators
There are decorators that return a callable that not calls the function that was given as an argument, but also do some other things, and therefore change the behaviour of that function.
If possible, yes. So that there is a standardized way to access changing variables, or to put limits on the content of the variable, similar to the @accepts and @produces decorators that are seen here ( https://wiki.python.org/moin/PythonDecoratorLibrary#Type_Enforcement_.28acce... )
On 04/01/2016 11:08 AM, Matthias welp wrote:
Even earlier, Ethan Furman wrote:
I don't see it happening. Making that change would be a lot of work, and the advantages (if any) of the second method over the first do not warrant it. -- ~Ethan~
On Fri, Apr 1, 2016 at 2:09 PM Matthias welp <boekewurm@gmail.com> wrote:
There is a standardized way. You can extend ``property`` or mimic its implementation. Then instead of class Foo: a = property(getter, no_cycles) You can write class Foo: a = NoCycles() I haven't linked to a how-to for doing this is, because I think it's unnecessary for most small projects. Every so often someone asks for a more pleasant syntax for specifying a property, getter and setter. Guido seems to consistently reply that he thinks our current situation is good enough. I'd dig up a link to the email archive for you, but Google wasn't being very kind to me.
On Fri, Apr 01, 2016 at 08:08:54PM +0200, Matthias welp wrote:
But decoratoring a *name binding* isn't going to do that. All it will do is limit the *initial value*, exactly as the function call does. After calling a = Char(length=10, value='empty') the name "a" is bound to the result of Char(...), whatever that happens to call. But that doesn't change the behaviour of the *name* "a", it only sets the value it is bound to. Nothing stops anyone from saying: a = 42 and re-binding the name to another value which is no longer a Char. Python has no default support for running custom code when binding arbitrary names to a value. To get this sort of thing to work, you are limited to attributes, using the descriptor protocol. I'm not going to explain descriptors now, you can google them, but property is a descriptor. So let's imagine that Python allows the @ syntax as you request, and go through the cases to see what that would imply. For local variables inside functions, or global top-level module variables, it doesn't give you any interesting power at all. All you have is an alternative syntax: @spam @eggs @cheese x = 999 is exactly the same as x = spam(eggs(cheese(999))) right now. You don't even save any characters: 28 (including newlines) for both. But this doesn't give us anything new and exciting, since x is now just a regular variable that can be replaced with some other value. So in this scenario, this sounds boring -- it gives you nothing you don't already have. Inside a class, we have the power of descriptors available to us, so we can use them for computed attributes. (That's how property works, among many others.) So suppose we have: class MyClass(object): @decorate x = 999 instance = MyClass() Now instance.x can be a descriptor, which means that it can enforce type and value validation rules, or logging, or whatever amazing functionality you want to add. This is good. But you can already do this. You just have to write: class MyClass(object): x = decorate(999) instead. If decorate() returns a descriptor, it returns a descriptor whatever syntax you use. What benefit do you gain? Well, in the function and class decorator case, you gain the benefit that the decoration is close to the class or function header, and you don't have to write the name three times: class MyClass(object): def method(self): [... many, many, many lines of code ...] method = decorate(method) But this isn't an advantage in the case of the variable: class MyClass(object): x = decorate(value) You only have to write the name once, and the decoration is not going to be far away. So just like the global variable case, there's no advantage to @decorate syntax, regardless of whether you are in a class or not. -- Steve
This is exactly my point. When they implemented descriptors (2.2) they did that to add to the new-style class system. I think they did a great job, but namespaces were overlooked in my opinion. Why should we be limited to class attributes when using descriptors? Any namespace which contains a '.__dict__' content mapping should be able to hold descriptors in my opinion. If this has been discussed before, then please link me to the relevant discussion, I'd love to read the points made.
But this isn't an advantage in the case of the variable
Assigning variables from function results is fairly common. When chaining function calls to get the desired behaviour of the variable, it will get confusing which part is the 'value' part, and which part is the 'behaviour' part, apart from namings: var = logging( require(int)( factorize( 4 )))) would get more clear if you wrote it this way: @logging @require(int) var = factorize(4) Something I just thought about: currently you can use the property() call to make an attribute have descriptor properties. This may be somewhat controversial, but maybe limit this to decorators only? While keeping the '.__dict__' override method open, the way things work currently won't change that much, but it will make the assignment of attributes or variables with descriptor properties a lot more intuitive, as you do not set a *name* to a variable and then can undo that just a while later: var = property(setter, getter, deleter, docs) var = 20 currently changes behaviour depending on what kind of scope it is located in (class description, any other scope), while decorators (for functions at least) work in every scope I can think of. I think that is strange, and that it should just be the same everywhere. Using decorators here could be a very nice and interesting choice.
Sorry, that was untested code. My expectations of class definitions was wrong, as it does not actually change behaviour inside it's own scope. I thought that when you are defining a class, that when you assign a property value to an attribute, that the attribute 'name value' will directly change it's behaviour to include the descriptor properties of the property object assigned. My mistake. On 3 April 2016 at 08:32, Chris Angelico <rosuav@gmail.com> wrote:
On Sun, Apr 3, 2016 at 4:53 PM, Matthias welp <boekewurm@gmail.com> wrote:
Ah. There is a significant difference between assignment within a class definition and assignment from a function _inside_ that class definition, but in any given scope, double assignment always does the same thing: last one wins. Which is a good thing, when it comes to the @property decorator: class LifeAndUniverse: @property def answer(self): return 42 @answer.setter def answer(self, value): print("No fair changing the answer!") @answer.deleter def answer(self): print("You just deleted.... everything.") Each function definition overwrites the previous "answer" with a new one, which (thanks to the way setter and deleter are implemented) incorporates the previous code, but nothing in Python mandates that. So is there anything left of the assignment-decorator proposal, or is it completely withdrawn? (I always like to read over even the bad proposals - there's often something good in them, Martin Farquhar Tupper's "Proverbial Philosophy" aside.) ChrisA
So is there anything left of the assignment-decorator proposal, or is it completely withdrawn?
I think this sums the current open ends up: - Namespace variables decoration was dismissed by one of Steven's posts, but is actually something that might be wanted (via being able to put descriptors into namespaces that have a __dict__ accessor (e.g. modules)) - Variable decoration can be more clear about descriptor/value difference at assignment - Giving property objects access to their variable's name (e.g. via __name__) like functions have would open up quite a bit of possibilities, and would mean decorators would get quite a bit more power than what they have. Something that I had said earlier, but what went on a sidepath - Decorators may directly *deep* set the behaviour of the variable, and with it set the further behaviour of the variable (in the same scope). Such that @decorator var = 20 var = 40 will not reset var to 40, but the var = 40 goes through the descriptor (if applied).
On Sun, Apr 3, 2016 at 6:02 PM, Matthias welp <boekewurm@gmail.com> wrote:
None of this is possible with decorators *per se*, as they simply mutate an object as it goes past. The magic you're seeing (and yearing for) comes from two places: functions have names (so the bit in "def SOMETHING(...)" actually affects the object constructed, not just the name it's bound to), and attribute access and the descriptor protocol. The latter is what makes @property work, but you don't need decorators to use descriptor protocol - in fact, it's crucial to the way bound methods are created. So, let's take your proposals one by one.
You'll need to elaborate on exactly what this can and can't do.
- Variable decoration can be more clear about descriptor/value difference at assignment
All Python names are in namespaces, so I'm not sure how this is different from the above.
This is fundamentally not possible in the general case; however, you could easily make a special form of the @property decorator which *requires* that the attribute name be the same as the function name that implements it, and then it would work (because function names are attributes of function objects, while "the name this is bound to" is not an attribute of anything). It's already accessible, as ClassName.property_name.fget.__name__; you could make this more accessible or more discoverable by subclassing property. In fact, you can even go meta:
However, this is vulnerable to the same thing that any other "what is my name" system has: you can rebind anything.
So there's fundamentally no way to say "what name is bound to me"; but you can quite effectively say "what is my canonical name", depending on a function to provide that name. ChrisA
On Sun, Apr 03, 2016 at 10:02:55AM +0200, Matthias welp wrote:
It's not that I dismissed the idea, but that such a thing is not possible today. Maybe you could hack up something tricky by replacing the module __dict__ with a subclass, or replacing the entire module object with some custom class instance, but that's messy and fragile, and I'm not sure that it would work for (say) globals inside functions. I think the idea of "namespace descriptors" is promising. But it probably needs a PEP and a significant amount of work to determine what actually is possible now and what needs support from the compiler.
- Variable decoration can be more clear about descriptor/value difference at assignment
I'm not sure what you mean by that.
- Giving property objects access to their variable's name (e.g. via __name__) like functions have would open up quite a bit of possibilities,
Such as what?
and would mean decorators would get quite a bit more power than what they have.
What extra power are you thinking of?
This functionality has nothing to do with decorators. If it were possible, if namespaces (other than classes) supported descriptors, then the decorator syntax doesn't add anything to this. You could write: var = magic(20) I'm not calling it "decorator" because the whole @ decorator syntax is irrelevant to this. With respect Matthias, I think you are conflating and mixing up the power of descriptors as applied to classes, and the separate and distinct powers of decorators. You don't need one to have the other. -- Steve
On Saturday, April 02, 2016 23:33, Chris Angelico wrote:
I can sort of see it, like this? class C(): def __init__(self): self.x = 99 @property def f(self): return self.x x = 101 @property def f(namespace): return namespace.x c = C() print(c.x) 99 print(c.f) 99 print(C.f.fget(c)) 99 print(x) 101 Here's the inconsistency, should implicitly bind to a "global namespace object". print(f) <property object at 0x00000000023FB6D8> In other words, something like this: class GNS: def __getattr__(self, name): prop = globals().get(name) if isinstance(prop, property): return prop.fget(self) return prop global_namespace = GNS() print(x is global_namespace.x, "should be 'True'") True should be 'True' print(global_namespace.f) 101
On Fri, Apr 01, 2016 at 07:44:21PM +0200, Michel Desmoulin wrote:
Not quite. It's actually: def bar(): pass bar = foo(bar) which is *three* uses of the same name, two of which may be a long way from the first. As you say:
But, I think the benefit for @decorator on functions is mainly because a function body is big,
That is certainly part of the justification for @ syntax. It might help to remember that decorators themselves have been possible in Python going all the way back to Python 1.5, at least, if not earlier, and the first three decorators in the standard library (property, classmethod and staticmethod) were added a few releases before decorator syntax using @. (If I remember correctly, property etc were added in 2.2 but @ syntax wasn't added until 2.4.) So the decorator concept had many years to prove itself before being given special syntax. The two reasons for adding extra syntax were: (1) to keep the decoration near the function declaration; and (2) to avoid having to repeat the function name three times.
I'm afraid I don't understand what you mean by this. What does "while on a variable" mean here? -- Steve
Steven D'Aprano writes:
Missing punctuation (see substring in square brackets): and this way we can read the decorator next to the function signature[. W]hile on a variable, this just add another way to call a function on a variable. So, it appears that he's proposing pure syntactic sugar with no semantic change at all. The syntactic effects are that decorator syntax separates function calls from the values they act on (-1 on that), and makes it appear that in Python a function can change the value of its formal argument (-1 on that, too). The latter characteristic may need some expansion. Of course Python has mutable objects which can be changed "inside" a function and that effect propagates to "outside" the function. But in the containing scope, while the object has mutated, the variable (more precisely, the name-object binding) has not. It seems to me that everything he's asked for can be accomplished by defining a descriptor class and assigning an instance of that class. I don't yet understand why that isn't good enough, unless it's just a matter of taste about the sweetness of syntax.
participants (9)
-
Chris Angelico -
Eric Fahlgren -
Ethan Furman -
Ian Kelly -
Matthias welp -
Michael Selik -
Michel Desmoulin -
Stephen J. Turnbull -
Steven D'Aprano