Hi folks I was thinking about how sometimes, a function sometimes acts on classes, and behaves very much like a method. Adding new methods to classes existing classes is currently somewhat difficult, and having pseudo methods would make that easier. Code example: (The syntax can most likely be improved upon) def has_vowels(self: str): for vowel in ["a", "e,", "i", "o", "u"]: if vowel in self: return True This allows one to wring `string.has_vowels()` instead of `has_vowels(string)`, which would make it easier to read, and would make it easier to add functionality to existing classes, without having to extend them. This would be useful for builtins or imported libraries, so one can fill in "missing" methods. * Simple way to extend classes * Improves readability * Easy to understand ~Paul
On 4 August 2017 at 08:39, Paul Laos <paul_laos@outlook.com> wrote:
Adding new methods to classes is deliberately (somewhat) difficult, as it makes it harder to locate the definition of a method. If you need to see the code for a method, you'd expect to look in the class definition. Making it common for people to put method definitions outside the class definition harms supportability by breaking that assumption.
That's very much a subjective view. Personally, I don't see "string.has_vowels()" as being any easier to read - except in the sense that it tells me that I can find the definition of has_vowels in the class definition of str (and I can find its documentation in the documentation of the str type). And your proposal removes this advantage!
This is a common technique in other languages like Ruby, but is considered specialised and somewhat of an advanced technique (monkeypatching) in Python. As you say yourself, the syntax will make it *easier* to do this - it's already possible, so the change doesn't add any new capabilities. Adding new syntax to the language typically needs a much stronger justification (either in terms of enabling fundamentally new techniques, or providing a significantly more natural spelling of something that's widely used and acknowledged as a common programming idiom). Sorry, but I'm -1 on this change. It doesn't let people do anything they can't do now, on the contrary it makes it simpler to use a technique which has readability and supportability problems, which as a result will mean that people will be inclined to use the approach without properly considering the consequences. Paul
Hi Paul, and welcome! On Fri, Aug 04, 2017 at 07:39:56AM +0000, Paul Laos wrote:
I'm not really sure what you mean by "acts on classes". I can only think of a function which takes a class as a parameter, and modifies the class. Like a class decorator. Or possibly a classmethod. But that's not what you seem to mean below. So I'm not quite certain I understand your proposal.
Adding new methods to classes existing classes is currently somewhat difficult,
If the class is written in Python, it isn't difficult at all, it is trivially easy. First define your method: def method(self, arg): pass Then inject it onto the class using ordinary attribute assignment: TheClass.method = method And we're done! If the class is a built-in, or otherwise written in C, then "somewhat difficult" is an understatement. I think it can't be done at all.
and having pseudo methods would make that easier.
I'm not sure that "easier" in this case would be better.
How does Python, and for that matter the human reader, know which class or classes that method is injected into? My guess is it looks at the annotation. But that's a big change: annotations are currently guaranteed to have no runtime semantics (apart from being stored in the function's __annotation__ attribute). I'm not saying that can't be done, but there may be consequences we haven't thought of. If we say dir(str), will "has_vowels" show up? How about vars(str)? How does this interact with metaclasses?
This allows one to wring `string.has_vowels()` instead of `has_vowels(string)`, which would make it easier to read,
Well that's one opinion.
http://www.virtuouscode.com/2008/02/23/why-monkeypatching-is-destroying-ruby... I think monkeypatching is great, so long as I'm the only one that does it. When other people do it, invariably they introduce bugs into my code by monkeypatching other things I didn't expect to be monkeypatched.
I'll agree with the first one of those, if by "simple" you mean "somebody else did all the work to make this syntax do what I want it to do". The work behind the scenes is not likely to be simple: for starters, allowing monkeypatching of built-ins is likely going to require a rather big re-design of the Python interpreter. -- Steve
Had not this been discussed here earlier this year? (And despite there being perceived dangers to readability in the long term, was accepted?) Here it is on an archive: https://mail.python.org/pipermail/python-ideas/2017-February/044551.html And anyway - along that discussion, despite dislikng the general idea, I got convinced that creating an outside method that makes "super" or "__class__" work was rather complicated. Maybe we could just have a decorator for that, that would properly create the __class__ cell? js -><- On 4 August 2017 at 08:32, Steven D'Aprano <steve@pearwood.info> wrote:
On 4 August 2017 at 10:31, Paul Moore <p.f.moore@gmail.com> wrote:
Nonetheless, a third party module with some decorators to allow doing that "the right way" might be usefull. If one is willing to write, or retrieve a candidate for that. (I don´ t think it is possible to inject the __class__ cell in a clean way, though) js -><-
Paul
On Fri, Aug 04, 2017 at 10:20:55AM -0300, Joao S. O. Bueno wrote:
I don't read this as the same proposal. For starters, I don't believe that it was intended to allow monkey-patching of builtins. Another is that the syntax is much more explicit about where the method is going: def MyClass.method(self, arg): ... is clearly a method of MyClass. There was, if I recall, some open discussion of whether arbitrary assignment targets should be allowed: def module.func(x or None)[23 + n].attr.__type__.method(self, arg): ... or if we should intentionally limit the allowed syntax, like we do for decorators. My vote is for intentionally limiting it to a single dotted name, like MyClass.method.
Complicated is an understatement. It's horrid :-) Here's the problem: we can successfully inject methods into a class: # -----%<----- class Parent: def spam(self): return "spam" class Child(Parent): def food(self): return 'yummy ' + self.spam() c = Child() c.food() # returns 'yummy spam' as expected # inject a new method def spam(self): return 'spam spam spam' Child.spam = spam c.food() # returns 'yummy spam spam spam' as expected # -----%<----- But not if you use the zero-argument form of super(): # -----%<----- del Child.spam # revert to original def spam(self): s = super().spam() return ' '.join([s]*3) Child.spam = spam c.food() # -----%<----- This raises: RuntimeError: super(): __class__ cell not found This is the simplest thing I've found that will fix it: # -----%<----- del Child.spam # revert to original again def outer(): __class__ = Child def spam(self): s = super().spam() return ' '.join([s]*3) return spam Child.spam = outer() c.food() # returns 'yummy spam spam spam' as expected # -----%<----- It's probably possibly to wrap this up in a decorator that takes Child as argument, but I expect it will probably require messing about with the undocumented FunctionType constructor to build up a new closure from the bits and pieces scavenged from the decorated function.
Maybe we could just have a decorator for that, that would properly create the __class__ cell?
I expect its possible. A challenge to somebody who wants to get their hands dirty. -- Steve
On 7 August 2017 at 18:48, Victor Stinner <victor.stinner@gmail.com> wrote:
Right, Python's opinionated design guidance is to clearly distinguish between "data first" designs using methods on objects and "algorithm first" designs using functools.singledispatch (or similar mechanisms), since they place different constraints on how new implementations are added, and where you should look for more information about how an algorithm works. Part of the intent behind this guidance is to better enable local reasoning about a piece of code: from my_string_utils import has_vowels if has_vowels(input("Enter a word: ")): print("Contains vowels!") else: print("Does not contain vowels!") Here, it is clear that if we want to know more about what "has_vowels" does, or if we want to request changes to how it works, then "my_string_utils" is where we need to go next. By contrast, that's significantly less clear if our string utils module were to implicitly modify the behaviour of input() or builtin strings: import my_string_utils if input("Enter a word: ").has_vowels(): print("Contains vowels!") else: print("Does not contain vowels!") To analyse and investigate this code, we need to "just know" that: - the result of "input()" doesn't normally have a "has_vowels()" method - therefore, importing "my_string_utils" must have either replaced the input builtin or mutated the str type - therefore, "my_string_utils" is probably the place to go for more information on "has_vowels" If our import line had instead looked like "import my_string_utils, my_other_utils", we'd have to go look at both of them to figure out where the "has_vowels()" method might be coming from (and hope it wasn't happening further down as a side effect of one of the modules *they* imported). Injecting methods rather than writing functions that dispatch on the type of their first argument also creates new opportunities for naming conflicts: while "my_string_utils.has_vowels" and "your_string_utils.has_vowels" can happily coexist in the same program without conflicts, there's only one "input" builtin, and only one "str" builtin. Can this level of explicitness be an obstacle at times? Yes, it can, especially for testing and interactive use, which is why Python offers features like wildcard imports, runtime support for monkeypatching of user-defined types, and runtime support for dynamically replacing builtins and module globals. However, the concerns around the difficulties of complexity management in the face of implicit action at a distance remain valid, so those features all fall into the category of "supported, but not encouraged, except in specific circumstances". Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Nick Coghlan writes:
To analyse and investigate this code, we need to "just know" that:
You can of course hope that help(input().has_vowels) will tell you where to find it. If it doesn't, well, shame on you for depending on source-unavailable software that you don't understand. ;-) I'm with you on implementing this feature; I don't like it. But I don't think the discoverability situation is as dire as you suggest.
On 9 August 2017 at 18:19, Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
We can't run "help" when we're reviewing a diff or otherwise reading code in a situation where interactive help isn't available :) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 4 August 2017 at 08:39, Paul Laos <paul_laos@outlook.com> wrote:
Adding new methods to classes is deliberately (somewhat) difficult, as it makes it harder to locate the definition of a method. If you need to see the code for a method, you'd expect to look in the class definition. Making it common for people to put method definitions outside the class definition harms supportability by breaking that assumption.
That's very much a subjective view. Personally, I don't see "string.has_vowels()" as being any easier to read - except in the sense that it tells me that I can find the definition of has_vowels in the class definition of str (and I can find its documentation in the documentation of the str type). And your proposal removes this advantage!
This is a common technique in other languages like Ruby, but is considered specialised and somewhat of an advanced technique (monkeypatching) in Python. As you say yourself, the syntax will make it *easier* to do this - it's already possible, so the change doesn't add any new capabilities. Adding new syntax to the language typically needs a much stronger justification (either in terms of enabling fundamentally new techniques, or providing a significantly more natural spelling of something that's widely used and acknowledged as a common programming idiom). Sorry, but I'm -1 on this change. It doesn't let people do anything they can't do now, on the contrary it makes it simpler to use a technique which has readability and supportability problems, which as a result will mean that people will be inclined to use the approach without properly considering the consequences. Paul
Hi Paul, and welcome! On Fri, Aug 04, 2017 at 07:39:56AM +0000, Paul Laos wrote:
I'm not really sure what you mean by "acts on classes". I can only think of a function which takes a class as a parameter, and modifies the class. Like a class decorator. Or possibly a classmethod. But that's not what you seem to mean below. So I'm not quite certain I understand your proposal.
Adding new methods to classes existing classes is currently somewhat difficult,
If the class is written in Python, it isn't difficult at all, it is trivially easy. First define your method: def method(self, arg): pass Then inject it onto the class using ordinary attribute assignment: TheClass.method = method And we're done! If the class is a built-in, or otherwise written in C, then "somewhat difficult" is an understatement. I think it can't be done at all.
and having pseudo methods would make that easier.
I'm not sure that "easier" in this case would be better.
How does Python, and for that matter the human reader, know which class or classes that method is injected into? My guess is it looks at the annotation. But that's a big change: annotations are currently guaranteed to have no runtime semantics (apart from being stored in the function's __annotation__ attribute). I'm not saying that can't be done, but there may be consequences we haven't thought of. If we say dir(str), will "has_vowels" show up? How about vars(str)? How does this interact with metaclasses?
This allows one to wring `string.has_vowels()` instead of `has_vowels(string)`, which would make it easier to read,
Well that's one opinion.
http://www.virtuouscode.com/2008/02/23/why-monkeypatching-is-destroying-ruby... I think monkeypatching is great, so long as I'm the only one that does it. When other people do it, invariably they introduce bugs into my code by monkeypatching other things I didn't expect to be monkeypatched.
I'll agree with the first one of those, if by "simple" you mean "somebody else did all the work to make this syntax do what I want it to do". The work behind the scenes is not likely to be simple: for starters, allowing monkeypatching of built-ins is likely going to require a rather big re-design of the Python interpreter. -- Steve
Had not this been discussed here earlier this year? (And despite there being perceived dangers to readability in the long term, was accepted?) Here it is on an archive: https://mail.python.org/pipermail/python-ideas/2017-February/044551.html And anyway - along that discussion, despite dislikng the general idea, I got convinced that creating an outside method that makes "super" or "__class__" work was rather complicated. Maybe we could just have a decorator for that, that would properly create the __class__ cell? js -><- On 4 August 2017 at 08:32, Steven D'Aprano <steve@pearwood.info> wrote:
On 4 August 2017 at 10:31, Paul Moore <p.f.moore@gmail.com> wrote:
Nonetheless, a third party module with some decorators to allow doing that "the right way" might be usefull. If one is willing to write, or retrieve a candidate for that. (I don´ t think it is possible to inject the __class__ cell in a clean way, though) js -><-
Paul
On Fri, Aug 04, 2017 at 10:20:55AM -0300, Joao S. O. Bueno wrote:
I don't read this as the same proposal. For starters, I don't believe that it was intended to allow monkey-patching of builtins. Another is that the syntax is much more explicit about where the method is going: def MyClass.method(self, arg): ... is clearly a method of MyClass. There was, if I recall, some open discussion of whether arbitrary assignment targets should be allowed: def module.func(x or None)[23 + n].attr.__type__.method(self, arg): ... or if we should intentionally limit the allowed syntax, like we do for decorators. My vote is for intentionally limiting it to a single dotted name, like MyClass.method.
Complicated is an understatement. It's horrid :-) Here's the problem: we can successfully inject methods into a class: # -----%<----- class Parent: def spam(self): return "spam" class Child(Parent): def food(self): return 'yummy ' + self.spam() c = Child() c.food() # returns 'yummy spam' as expected # inject a new method def spam(self): return 'spam spam spam' Child.spam = spam c.food() # returns 'yummy spam spam spam' as expected # -----%<----- But not if you use the zero-argument form of super(): # -----%<----- del Child.spam # revert to original def spam(self): s = super().spam() return ' '.join([s]*3) Child.spam = spam c.food() # -----%<----- This raises: RuntimeError: super(): __class__ cell not found This is the simplest thing I've found that will fix it: # -----%<----- del Child.spam # revert to original again def outer(): __class__ = Child def spam(self): s = super().spam() return ' '.join([s]*3) return spam Child.spam = outer() c.food() # returns 'yummy spam spam spam' as expected # -----%<----- It's probably possibly to wrap this up in a decorator that takes Child as argument, but I expect it will probably require messing about with the undocumented FunctionType constructor to build up a new closure from the bits and pieces scavenged from the decorated function.
Maybe we could just have a decorator for that, that would properly create the __class__ cell?
I expect its possible. A challenge to somebody who wants to get their hands dirty. -- Steve
On 7 August 2017 at 18:48, Victor Stinner <victor.stinner@gmail.com> wrote:
Right, Python's opinionated design guidance is to clearly distinguish between "data first" designs using methods on objects and "algorithm first" designs using functools.singledispatch (or similar mechanisms), since they place different constraints on how new implementations are added, and where you should look for more information about how an algorithm works. Part of the intent behind this guidance is to better enable local reasoning about a piece of code: from my_string_utils import has_vowels if has_vowels(input("Enter a word: ")): print("Contains vowels!") else: print("Does not contain vowels!") Here, it is clear that if we want to know more about what "has_vowels" does, or if we want to request changes to how it works, then "my_string_utils" is where we need to go next. By contrast, that's significantly less clear if our string utils module were to implicitly modify the behaviour of input() or builtin strings: import my_string_utils if input("Enter a word: ").has_vowels(): print("Contains vowels!") else: print("Does not contain vowels!") To analyse and investigate this code, we need to "just know" that: - the result of "input()" doesn't normally have a "has_vowels()" method - therefore, importing "my_string_utils" must have either replaced the input builtin or mutated the str type - therefore, "my_string_utils" is probably the place to go for more information on "has_vowels" If our import line had instead looked like "import my_string_utils, my_other_utils", we'd have to go look at both of them to figure out where the "has_vowels()" method might be coming from (and hope it wasn't happening further down as a side effect of one of the modules *they* imported). Injecting methods rather than writing functions that dispatch on the type of their first argument also creates new opportunities for naming conflicts: while "my_string_utils.has_vowels" and "your_string_utils.has_vowels" can happily coexist in the same program without conflicts, there's only one "input" builtin, and only one "str" builtin. Can this level of explicitness be an obstacle at times? Yes, it can, especially for testing and interactive use, which is why Python offers features like wildcard imports, runtime support for monkeypatching of user-defined types, and runtime support for dynamically replacing builtins and module globals. However, the concerns around the difficulties of complexity management in the face of implicit action at a distance remain valid, so those features all fall into the category of "supported, but not encouraged, except in specific circumstances". Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Nick Coghlan writes:
To analyse and investigate this code, we need to "just know" that:
You can of course hope that help(input().has_vowels) will tell you where to find it. If it doesn't, well, shame on you for depending on source-unavailable software that you don't understand. ;-) I'm with you on implementing this feature; I don't like it. But I don't think the discoverability situation is as dire as you suggest.
On 9 August 2017 at 18:19, Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
We can't run "help" when we're reviewing a diff or otherwise reading code in a situation where interactive help isn't available :) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (8)
-
Antoine Rozo -
Joao S. O. Bueno -
Nick Coghlan -
Paul Laos -
Paul Moore -
Stephen J. Turnbull -
Steven D'Aprano -
Victor Stinner