Decorator to avoid a mistake
Hi there, I like python easy extend class and reusse code, but sometime I overwrite involontary some functions or variables from parents. I think about a decorator to confirm overwrite is intended and put a warning if is not present. class A: def hello(self): print('Hello A') class B: def hello(self): print('Hello B') class C: @overwrite def hello(self): print('Hello C') b=B() c=C() b.hello() Warning overwrite hello method... Hello B c.hello() Hello C Dont know if this should be add to language this way or another, or if this should be in pylint only ... Perhaps someone get better way to achieve this or something already exist? May the python be with you, Regards
This is just a convenience utility that would impact performance. This kind of enhancements, in my opinion, should be taken care by the IDEs, not by the interpreters.
I think you could implement this yourself with metaclasses and it wouldn't have much (if any) performance hit per-call or per-instantiation (just a bit slower when creating the class definition). It's a bit heavy-handed-hand-holding–if you ask me–but if you want to do it, the language gives you the ability. On Tue, Nov 22, 2016 at 3:24 PM, Adrián Orive Oneca < adrian.orive.oneca@gmail.com> wrote:
On 23 November 2016 at 08:08, François Leblanc <fleblanc50@gmail.com> wrote:
The point here is that if there is a way to get it without a performance cost (I can't imagine there would be, it's got to add a check for whether you're overriding a method at a minimum, but you may be able to come up with something) then you can do that within the language as it stands. It may need a specialised metaclass, but that's not a problem. At least, if it *is* an issue to you, you can argue for having it be the behaviour of the default metaclass once you've demonstrated that it works and doesn't hurt performance. And if there is a performance hit, then having the behaviour as opt-in based on a custom metaclass means that people can choose if they want to pay that cost. Paul
I can imagine using a metaclass specialized witch can be activate or desactivate but the cost of decorator call still be here... I think its will be a good improvement if we can provide a solution for this, and i ask myself if this can be set in interpreter with a flag to activate for exemple and with no performance cost without the flag set, so you will have the choice and a good way to check youre code. To have no performance cost perhaps the best way it's to use docstring and pylint... 2016-11-23 10:10 GMT+01:00 Paul Moore <p.f.moore@gmail.com>:
Related thread https://mail.python.org/pipermail/python-ideas/2016-July/041095.html On Nov 23, 2016 20:30, "François Leblanc" <fleblanc50@gmail.com> wrote:
Similar or related issue recently open and quickly closed: http://bugs.python.org/issue28776 "Duplicate method names should be an error" In short, such job should be done by linters. I'm quite sure that many of them already implement such check. Victor
You can do it at run-time, if you so desire, without a measurable performance hit with a metaclass. Here's a hacky demo: https://gist.github.com/nicktimko/5f08d6adfa1dbe1319c3bfc715ec0aa4#file-over... (Pedants: Any performance hit will be constant-time and probably less than a stray import that you don't need. If you're worried about optimizing constant-time things, I think you have larger problems.) On Thu, Nov 24, 2016 at 5:48 PM, Victor Stinner <victor.stinner@gmail.com> wrote:
On Fri, Nov 25, 2016 at 02:21:28PM -0500, Nick Timkovich wrote:
All I get at that page is "Sorry, something went wrong." Don't you love informative error messages?
time.sleep(360000) is constant time *wink* Sorry I couldn't resist... But seriously... this sort of check belongs in a linter, not in the core language, because its not necessarily an error or a mistake to override an existing method. In subclasses, the ability to create a method with the same name as one in a parent class is fundamental to how inheritence works, and even within a single class there's a use (admittedly uncommon) for re-using the same name: class Spam: def method(self): ... if condition: method = wrapper(method) The down-side of this flexibility and power is that more responsibility is placed in the hands of the programmer, which is hard on beginners. Sometimes I think Python-as-a-teaching-language and Python-as-a- production-language are strongly opposed. I wonder whether there might be a case to be made for a --with-training-wheels option? But that's better placed in the IDE, not the core language. I think this request is perhaps better suited as a feature request for IDLE rather than a language feature. -- Steve
This idea is being kicked around from forum to forum and nobody wants to have it. Here it's brought up from time to time and the response is usually "let a linter do it". In mypy (which is essentially a powerful linter) it was proposed ( https://github.com/python/mypy/issues/1888) and the response was essentially "better go to python-ideas or bugs.python.org". It's also been proposed as a PEP 484 feature: https://github.com/python/typing/issues/269#issuecomment-243765549 . I think one reason why such proposals are unwelcome to experienced users may be that when done right this is totally legitimate, and the requirement to use an @override decorator is just making code more verbose with very little benefit. (I could see a benefit -- if this is used consistently it could make spelunking a large codebase easier, because you would know which methods are overrides. Something like mypy could then enforce its use. But an IDE could also just highlight method overrides differently, maybe PyCharm or PyDev already do that?) -- --Guido van Rossum (python.org/~guido <http://python.org/%7Eguido>)
Here's hopefully a working link; GitHub's rendering seems to be iffy: https://nbviewer.jupyter.org/urls/gist.githubusercontent.com/nicktimko/5f08d... Knowing how to do some introspection with dunder magic attributes can make tracing methods easier. There's also a bunch of caveats that could probably bite you (and make my demo full of holes) On Fri, Nov 25, 2016 at 10:26 PM, Guido van Rossum <guido@python.org> wrote:
Le 26 nov. 2016 4:27 AM, "Guido van Rossum" <guido@python.org> a écrit :
This idea is being kicked around from forum to forum and nobody wants to
Here it's brought up from time to time and the response is usually "let a
have it. Sorry, I want to have it but I dont want to pay the price in performance cost.. linter do it". It's true, but in other hand because the subject come back from time to time, perhaps something better can be imagine the need is here. little benefit. (I could see a benefit -- if this is used consistently it could make spelunking a large codebase easier, because you would know which methods are overrides. Something like mypy could then enforce its use. But an IDE could also just highlight method overrides differently, maybe PyCharm or PyDev already do that?) The goal is not to make the code more verbose at all, it's to make code more sure in big or/and collaborative projects. The goal it's to give the possibility to secure the use of one of the more powerful feature of python. I think this kind of mistake may slow the time development and consequently afraid manager of using python in business environnement. Last time i have extend a Tkinter Top le vel and get in trouble with some comportement, I've just introduce a configuration function in my extended class. We often give same name for same functionnalities and that is the trouble if its not volontary. In case the function or variable is runtime defined its very hard to detect the error and pylint tools like will not help...
On 26 November 2016 at 13:26, Guido van Rossum <guido@python.org> wrote:
It's also the case that if we're going to provide decorators to help with class definitions, there are more valuable things that we can do beyond merely warning about accidental method and attribute overrides. For example, I've recently started using Hynek Schlawack's attrs project, which pairs up an "attr.attributes" class decorator with an "attr.attrib()" descriptor to make the following a class definition with totally ordered instances and a nice repr() implementation: @attributes class MyClass: attr1 = attrib() attr2 = attrib() ... It's *really* nice to use, and means I'm more inclined to define explicit helper classes for interim results rather than passing convenient-but-hard-to-debug raw tuples, lists and dicts around. The docs summarise the special method implementations that the decorator will inject for you in https://attrs.readthedocs.io/en/stable/why.html#hand-written-classes and go into more details on how it works at https://attrs.readthedocs.io/en/stable/how-does-it-work.html While the focus of attrs itself is very much on defining data structures where composition is the main form of re-use rather than inheritance, subclassing *is* supported, and such a system could also be used to complain about accidental attribute and method name collisions. Integrating a check for accidental overrides with a method-boilerplate reduction class decorator like the one in attrs would help address a few potential objections: - undecorated class definitions would be unaffected, so there'd be no start-up performance hit for existing code - leaving out the decorator when you intended to use it would also mean you wouldn't have an __init__ method defined, so you'd notice pretty fast if you forgot to use it - for the cost of some explicit decorators on the class definition and any method overrides, you get to skip writing a whole set of special methods (__new__/__init__, __repr__/etc, __eq__/__ne__, __lt__/etc) The specific names from the attrs project wouldn't be suitable for the standard library, but something like the following might work: @autoinit class MyClass: attr1 = instanceattr() attr2 = instanceattr() Where "instanceattr" is inspired by "classmethod" and "staticmethod", while "autoinit" refers directly to the main benefit of the class decorator (i.e. it writes your __init__ method for you based on the "instanceattr" definitions). The standardised variant of that could be kept very simple, while more advanced features like data validation and conversion were left to third party libraries like attrs. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 26 November 2016 at 07:16, Nick Coghlan <ncoghlan@gmail.com> wrote:
This comment made me think about another potential issue with the @override proposal. If overriding a method without using the decorator were to produce a warning, how would that interact with custom class decorators or metaclasses that inject methods into a class? Would they have to take special care to detect and mark overrides? While it's true that if a custom metaclass injects an unintended override, that's just as much a problem as if the user explicitly writes one, the problem arises for the end user of such a custom metaclass, who will get an obscure warning about internals that they probably don't know how to fix. Nothing insurmountable, sure, but it does act as a reminder that a change like this could have pretty wide reaching implications... Paul
On 26 November 2016 at 21:15, Paul Moore <p.f.moore@gmail.com> wrote:
It would potentially be even more annoying than that - if we did something like this without being sufficiently careful about it, you'd need to use the decorator any time you overwrote a special method that's actually implemented on "object". Ditto for method overrides when inheriting from an abstract base class, or any other base class that defines an API where the base class method implementation raises NotImplementedError. By contrast, if it's opt-in via a class decorator, then folks that want additional definition time assistance from the interpreter can request that explicitly, while existing code remains unaffected. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
If we can sawp base class object with a flag It would be possible to use a special base object that could warn for this kind of problem? This can let write specifics metaclass witch replace base object by default, and when all is checked for end user don't use the flag... Does it is possible? What do you think about? Le 26/11/2016 à 12:26, Nick Coghlan a écrit :
On 26 November 2016 at 22:24, France3 <fleblanc50@gmail.com> wrote:
There's no need to do this in a base class, since it can be done via external introspection. That introspection could live in at least a couple of different places: - a class decorator, which checks for unmarked overrides at class definition time - a test utility, that given a module name, imports the module, finds any defined classes, and checks them for unmarked overrides All an override marker would have to do to enable that introspection is to set a particular attribute on the method definition. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Am I missing something? Given Python's dynamic nature, there is simply no way to know if a method is overriding a superclass' method until it is called -- and, now that I think about it even then you don't know. At compile time, none of the superclasses may have the given method. At run time, a method could be monkey patched into the superclass before the subclass' instance is created. In fact, at run time, the superclass could get the method added after instances are created. At calling time, the subclass' method will be found, and used, and the search stops there -- no way to know if there is one with the same name further up the MRO. This is simply incompatable with a language this dynamic. On Fri, Nov 25, 2016 at 7:26 PM, Guido van Rossum <guido@python.org> wrote:
Here it's brought up from time to time and the response is usually "let a linter do it".
Exactly -- it can't be enforced, but maybe it's nice for a linter to give some information at develop time. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Mon, Nov 28, 2016 at 10:11 AM, Chris Barker <chris.barker@noaa.gov> wrote:
Not so fast! Python is also so dynamic that you can easily create a metaclass (or a class decorator) that does the checking (assuming reasonable behavior of all classes involved) at class definition time. Given that the use of such a metaclass will be voluntary, for classes with unreasonable behavior it should simply not be applied. I'm not sure yet if it is desirable to have such a metaclass or class decorator in the stdlib, nor have I thought through its design. Such a design should first be deployed via PyPI and then we can judge its effectiveness based on how popular it becomes. (There may already be one, I've not done that research either.) -- --Guido van Rossum (python.org/~guido)
On Mon, Nov 28, 2016 at 10:22 AM, Guido van Rossum <guido@python.org> wrote:
but that's class definition time -- can't classes be dynamically mangled later on -- after subclasses have been defined? If I have this right the mro is set at class definition time, so it is knows which classes are in the tree -- but class objects themselves are mutable -- methods can be added later on. so a subclass could have a method that isn't overriding anything when it's defined, but ends up overriding something later on, 'cause the base class changed under it. Isn't this why the mro is searched at method calling time? rather than a static table being used? (if not -- wouldnt that be a nice optimization?) Granted -- this kind of late class mangling has got to be pretty unusual (and maybe only useful for mocking, or??) but it is there. Which is why it could be useful to have some syntax or convention for specifying "I'm intending to override a method", that linters or type checkers, or whatever external tool, can use. But it can't be enforced at runtime in the language. much like type annotations... -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Mon, Nov 28, 2016 at 1:32 PM, Chris Barker <chris.barker@noaa.gov> wrote:
They can, and they @override can be bypassed. I don't see that as a condemnation of @overload -- it just means that it's not perfect, which is fine with me (given that we're talking about monkey-patching here).
Depends on how much enforcement you need. Bike locks also don't typically enforce that your bike doesn't get stolen... -- --Guido van Rossum (python.org/~guido)
On Mon, Nov 28, 2016 at 1:37 PM, Guido van Rossum <guido@python.org> wrote:
sure -- but this all strikes me as kind of like type checking -- there is a lot of use for robust code, but we don't want it at run-time in the language. Also -- the ship has kinda sailed on this - maybe a @not_override would make more sense. Isn't the goal to make sure you don't accidentally override a method? saying "I know I'm overriding this" is less useful than "I'm not intending to override anything here" -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Mon, Nov 28, 2016 at 1:44 PM, Chris Barker <chris.barker@noaa.gov> wrote:
I think you're fighting a straw man. I never said @override should be added to the language. I said that it would be useful to have a 3rd party metaclass or a class decorator that implements it which packages may voluntarily use to constrain their subclasses (or their own uses -- different designs are possible). -- --Guido van Rossum (python.org/~guido)
On Mon, Nov 28, 2016 at 1:50 PM, Guido van Rossum <guido@python.org> wrote:
I know -- I just happened to add that to a reply to you... That was for the OP, or anyone else thinking of writing such a thing. And I still think it would better be added to a linting tool than at run-time -- but let whoever writes it figure that out. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On 29 November 2016 at 08:01, Chris Barker <chris.barker@noaa.gov> wrote:
Writing linting tools for Python class hierarchies is pretty hard in the general case, since you have to account for the fact that the module level control flow logic is Turing complete (and many linters simply don't try, instead saying "don't do that if you want the linter to work properly"). By contrast, a class decorator or metaclass can do the checks at runtime by walking the MRO, just as ABCMeta can look for @abstractmethod declarations that haven't been overridden in a subclass, and SQL Alchemy can map subclass inheritance to table linkages. If you look at the way attrs writes its automatic __init__ methods for example, it creates a single flat __init__ based on the definition time MRO: https://attrs.readthedocs.io/en/stable/how-does-it-work.html That's the nice part about this kind of thing being opt-in: the definition time tooling *doesn't* have to cope with the full extent of Python's dynamic nature, as it can expect the user to be cooperating with the tool to some degree, and "you're not cooperating" can be a valid error to throw. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Steven D'Aprano writes:
Sometimes I think Python-as-a-teaching-language and Python-as-a- production-language are strongly opposed.
I haven't found that to be the case. Linters are quite effective, as long as you discipline the students to use them. I don't think this check is in the linters I use, and if not I'd like to see it added, though.
I wonder whether there might be a case to be made for a --with-training-wheels option?
You could make it, but I can't imagine any argument that doesn't immediately fall prey to
But that's better placed in the IDE, not the core language.
Maybe my imagination is just poor, but I think the burden of proof is on those who would add --with-training-wheels to the implementation. (N.B. This is an implementation issue, not something that would be in the language spec! :-)
Hello, While I think this should not be "on by default", I don't see the harm in being able to opt-in to this behavior. I also figured spending a few minutes attempting to write this would be fun: https://gist.github.com/veloutin/2ec3e5246651f5de78442516d8e24fc1 François: sorry about the double reply, I forgot to reply to the list. Vince
This is just a convenience utility that would impact performance. This kind of enhancements, in my opinion, should be taken care by the IDEs, not by the interpreters.
I think you could implement this yourself with metaclasses and it wouldn't have much (if any) performance hit per-call or per-instantiation (just a bit slower when creating the class definition). It's a bit heavy-handed-hand-holding–if you ask me–but if you want to do it, the language gives you the ability. On Tue, Nov 22, 2016 at 3:24 PM, Adrián Orive Oneca < adrian.orive.oneca@gmail.com> wrote:
On 23 November 2016 at 08:08, François Leblanc <fleblanc50@gmail.com> wrote:
The point here is that if there is a way to get it without a performance cost (I can't imagine there would be, it's got to add a check for whether you're overriding a method at a minimum, but you may be able to come up with something) then you can do that within the language as it stands. It may need a specialised metaclass, but that's not a problem. At least, if it *is* an issue to you, you can argue for having it be the behaviour of the default metaclass once you've demonstrated that it works and doesn't hurt performance. And if there is a performance hit, then having the behaviour as opt-in based on a custom metaclass means that people can choose if they want to pay that cost. Paul
I can imagine using a metaclass specialized witch can be activate or desactivate but the cost of decorator call still be here... I think its will be a good improvement if we can provide a solution for this, and i ask myself if this can be set in interpreter with a flag to activate for exemple and with no performance cost without the flag set, so you will have the choice and a good way to check youre code. To have no performance cost perhaps the best way it's to use docstring and pylint... 2016-11-23 10:10 GMT+01:00 Paul Moore <p.f.moore@gmail.com>:
Related thread https://mail.python.org/pipermail/python-ideas/2016-July/041095.html On Nov 23, 2016 20:30, "François Leblanc" <fleblanc50@gmail.com> wrote:
Similar or related issue recently open and quickly closed: http://bugs.python.org/issue28776 "Duplicate method names should be an error" In short, such job should be done by linters. I'm quite sure that many of them already implement such check. Victor
You can do it at run-time, if you so desire, without a measurable performance hit with a metaclass. Here's a hacky demo: https://gist.github.com/nicktimko/5f08d6adfa1dbe1319c3bfc715ec0aa4#file-over... (Pedants: Any performance hit will be constant-time and probably less than a stray import that you don't need. If you're worried about optimizing constant-time things, I think you have larger problems.) On Thu, Nov 24, 2016 at 5:48 PM, Victor Stinner <victor.stinner@gmail.com> wrote:
On Fri, Nov 25, 2016 at 02:21:28PM -0500, Nick Timkovich wrote:
All I get at that page is "Sorry, something went wrong." Don't you love informative error messages?
time.sleep(360000) is constant time *wink* Sorry I couldn't resist... But seriously... this sort of check belongs in a linter, not in the core language, because its not necessarily an error or a mistake to override an existing method. In subclasses, the ability to create a method with the same name as one in a parent class is fundamental to how inheritence works, and even within a single class there's a use (admittedly uncommon) for re-using the same name: class Spam: def method(self): ... if condition: method = wrapper(method) The down-side of this flexibility and power is that more responsibility is placed in the hands of the programmer, which is hard on beginners. Sometimes I think Python-as-a-teaching-language and Python-as-a- production-language are strongly opposed. I wonder whether there might be a case to be made for a --with-training-wheels option? But that's better placed in the IDE, not the core language. I think this request is perhaps better suited as a feature request for IDLE rather than a language feature. -- Steve
This idea is being kicked around from forum to forum and nobody wants to have it. Here it's brought up from time to time and the response is usually "let a linter do it". In mypy (which is essentially a powerful linter) it was proposed ( https://github.com/python/mypy/issues/1888) and the response was essentially "better go to python-ideas or bugs.python.org". It's also been proposed as a PEP 484 feature: https://github.com/python/typing/issues/269#issuecomment-243765549 . I think one reason why such proposals are unwelcome to experienced users may be that when done right this is totally legitimate, and the requirement to use an @override decorator is just making code more verbose with very little benefit. (I could see a benefit -- if this is used consistently it could make spelunking a large codebase easier, because you would know which methods are overrides. Something like mypy could then enforce its use. But an IDE could also just highlight method overrides differently, maybe PyCharm or PyDev already do that?) -- --Guido van Rossum (python.org/~guido <http://python.org/%7Eguido>)
Here's hopefully a working link; GitHub's rendering seems to be iffy: https://nbviewer.jupyter.org/urls/gist.githubusercontent.com/nicktimko/5f08d... Knowing how to do some introspection with dunder magic attributes can make tracing methods easier. There's also a bunch of caveats that could probably bite you (and make my demo full of holes) On Fri, Nov 25, 2016 at 10:26 PM, Guido van Rossum <guido@python.org> wrote:
Le 26 nov. 2016 4:27 AM, "Guido van Rossum" <guido@python.org> a écrit :
This idea is being kicked around from forum to forum and nobody wants to
Here it's brought up from time to time and the response is usually "let a
have it. Sorry, I want to have it but I dont want to pay the price in performance cost.. linter do it". It's true, but in other hand because the subject come back from time to time, perhaps something better can be imagine the need is here. little benefit. (I could see a benefit -- if this is used consistently it could make spelunking a large codebase easier, because you would know which methods are overrides. Something like mypy could then enforce its use. But an IDE could also just highlight method overrides differently, maybe PyCharm or PyDev already do that?) The goal is not to make the code more verbose at all, it's to make code more sure in big or/and collaborative projects. The goal it's to give the possibility to secure the use of one of the more powerful feature of python. I think this kind of mistake may slow the time development and consequently afraid manager of using python in business environnement. Last time i have extend a Tkinter Top le vel and get in trouble with some comportement, I've just introduce a configuration function in my extended class. We often give same name for same functionnalities and that is the trouble if its not volontary. In case the function or variable is runtime defined its very hard to detect the error and pylint tools like will not help...
On 26 November 2016 at 13:26, Guido van Rossum <guido@python.org> wrote:
It's also the case that if we're going to provide decorators to help with class definitions, there are more valuable things that we can do beyond merely warning about accidental method and attribute overrides. For example, I've recently started using Hynek Schlawack's attrs project, which pairs up an "attr.attributes" class decorator with an "attr.attrib()" descriptor to make the following a class definition with totally ordered instances and a nice repr() implementation: @attributes class MyClass: attr1 = attrib() attr2 = attrib() ... It's *really* nice to use, and means I'm more inclined to define explicit helper classes for interim results rather than passing convenient-but-hard-to-debug raw tuples, lists and dicts around. The docs summarise the special method implementations that the decorator will inject for you in https://attrs.readthedocs.io/en/stable/why.html#hand-written-classes and go into more details on how it works at https://attrs.readthedocs.io/en/stable/how-does-it-work.html While the focus of attrs itself is very much on defining data structures where composition is the main form of re-use rather than inheritance, subclassing *is* supported, and such a system could also be used to complain about accidental attribute and method name collisions. Integrating a check for accidental overrides with a method-boilerplate reduction class decorator like the one in attrs would help address a few potential objections: - undecorated class definitions would be unaffected, so there'd be no start-up performance hit for existing code - leaving out the decorator when you intended to use it would also mean you wouldn't have an __init__ method defined, so you'd notice pretty fast if you forgot to use it - for the cost of some explicit decorators on the class definition and any method overrides, you get to skip writing a whole set of special methods (__new__/__init__, __repr__/etc, __eq__/__ne__, __lt__/etc) The specific names from the attrs project wouldn't be suitable for the standard library, but something like the following might work: @autoinit class MyClass: attr1 = instanceattr() attr2 = instanceattr() Where "instanceattr" is inspired by "classmethod" and "staticmethod", while "autoinit" refers directly to the main benefit of the class decorator (i.e. it writes your __init__ method for you based on the "instanceattr" definitions). The standardised variant of that could be kept very simple, while more advanced features like data validation and conversion were left to third party libraries like attrs. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 26 November 2016 at 07:16, Nick Coghlan <ncoghlan@gmail.com> wrote:
This comment made me think about another potential issue with the @override proposal. If overriding a method without using the decorator were to produce a warning, how would that interact with custom class decorators or metaclasses that inject methods into a class? Would they have to take special care to detect and mark overrides? While it's true that if a custom metaclass injects an unintended override, that's just as much a problem as if the user explicitly writes one, the problem arises for the end user of such a custom metaclass, who will get an obscure warning about internals that they probably don't know how to fix. Nothing insurmountable, sure, but it does act as a reminder that a change like this could have pretty wide reaching implications... Paul
On 26 November 2016 at 21:15, Paul Moore <p.f.moore@gmail.com> wrote:
It would potentially be even more annoying than that - if we did something like this without being sufficiently careful about it, you'd need to use the decorator any time you overwrote a special method that's actually implemented on "object". Ditto for method overrides when inheriting from an abstract base class, or any other base class that defines an API where the base class method implementation raises NotImplementedError. By contrast, if it's opt-in via a class decorator, then folks that want additional definition time assistance from the interpreter can request that explicitly, while existing code remains unaffected. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
If we can sawp base class object with a flag It would be possible to use a special base object that could warn for this kind of problem? This can let write specifics metaclass witch replace base object by default, and when all is checked for end user don't use the flag... Does it is possible? What do you think about? Le 26/11/2016 à 12:26, Nick Coghlan a écrit :
On 26 November 2016 at 22:24, France3 <fleblanc50@gmail.com> wrote:
There's no need to do this in a base class, since it can be done via external introspection. That introspection could live in at least a couple of different places: - a class decorator, which checks for unmarked overrides at class definition time - a test utility, that given a module name, imports the module, finds any defined classes, and checks them for unmarked overrides All an override marker would have to do to enable that introspection is to set a particular attribute on the method definition. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Am I missing something? Given Python's dynamic nature, there is simply no way to know if a method is overriding a superclass' method until it is called -- and, now that I think about it even then you don't know. At compile time, none of the superclasses may have the given method. At run time, a method could be monkey patched into the superclass before the subclass' instance is created. In fact, at run time, the superclass could get the method added after instances are created. At calling time, the subclass' method will be found, and used, and the search stops there -- no way to know if there is one with the same name further up the MRO. This is simply incompatable with a language this dynamic. On Fri, Nov 25, 2016 at 7:26 PM, Guido van Rossum <guido@python.org> wrote:
Here it's brought up from time to time and the response is usually "let a linter do it".
Exactly -- it can't be enforced, but maybe it's nice for a linter to give some information at develop time. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Mon, Nov 28, 2016 at 10:11 AM, Chris Barker <chris.barker@noaa.gov> wrote:
Not so fast! Python is also so dynamic that you can easily create a metaclass (or a class decorator) that does the checking (assuming reasonable behavior of all classes involved) at class definition time. Given that the use of such a metaclass will be voluntary, for classes with unreasonable behavior it should simply not be applied. I'm not sure yet if it is desirable to have such a metaclass or class decorator in the stdlib, nor have I thought through its design. Such a design should first be deployed via PyPI and then we can judge its effectiveness based on how popular it becomes. (There may already be one, I've not done that research either.) -- --Guido van Rossum (python.org/~guido)
On Mon, Nov 28, 2016 at 10:22 AM, Guido van Rossum <guido@python.org> wrote:
but that's class definition time -- can't classes be dynamically mangled later on -- after subclasses have been defined? If I have this right the mro is set at class definition time, so it is knows which classes are in the tree -- but class objects themselves are mutable -- methods can be added later on. so a subclass could have a method that isn't overriding anything when it's defined, but ends up overriding something later on, 'cause the base class changed under it. Isn't this why the mro is searched at method calling time? rather than a static table being used? (if not -- wouldnt that be a nice optimization?) Granted -- this kind of late class mangling has got to be pretty unusual (and maybe only useful for mocking, or??) but it is there. Which is why it could be useful to have some syntax or convention for specifying "I'm intending to override a method", that linters or type checkers, or whatever external tool, can use. But it can't be enforced at runtime in the language. much like type annotations... -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Mon, Nov 28, 2016 at 1:32 PM, Chris Barker <chris.barker@noaa.gov> wrote:
They can, and they @override can be bypassed. I don't see that as a condemnation of @overload -- it just means that it's not perfect, which is fine with me (given that we're talking about monkey-patching here).
Depends on how much enforcement you need. Bike locks also don't typically enforce that your bike doesn't get stolen... -- --Guido van Rossum (python.org/~guido)
On Mon, Nov 28, 2016 at 1:37 PM, Guido van Rossum <guido@python.org> wrote:
sure -- but this all strikes me as kind of like type checking -- there is a lot of use for robust code, but we don't want it at run-time in the language. Also -- the ship has kinda sailed on this - maybe a @not_override would make more sense. Isn't the goal to make sure you don't accidentally override a method? saying "I know I'm overriding this" is less useful than "I'm not intending to override anything here" -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Mon, Nov 28, 2016 at 1:44 PM, Chris Barker <chris.barker@noaa.gov> wrote:
I think you're fighting a straw man. I never said @override should be added to the language. I said that it would be useful to have a 3rd party metaclass or a class decorator that implements it which packages may voluntarily use to constrain their subclasses (or their own uses -- different designs are possible). -- --Guido van Rossum (python.org/~guido)
On Mon, Nov 28, 2016 at 1:50 PM, Guido van Rossum <guido@python.org> wrote:
I know -- I just happened to add that to a reply to you... That was for the OP, or anyone else thinking of writing such a thing. And I still think it would better be added to a linting tool than at run-time -- but let whoever writes it figure that out. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On 29 November 2016 at 08:01, Chris Barker <chris.barker@noaa.gov> wrote:
Writing linting tools for Python class hierarchies is pretty hard in the general case, since you have to account for the fact that the module level control flow logic is Turing complete (and many linters simply don't try, instead saying "don't do that if you want the linter to work properly"). By contrast, a class decorator or metaclass can do the checks at runtime by walking the MRO, just as ABCMeta can look for @abstractmethod declarations that haven't been overridden in a subclass, and SQL Alchemy can map subclass inheritance to table linkages. If you look at the way attrs writes its automatic __init__ methods for example, it creates a single flat __init__ based on the definition time MRO: https://attrs.readthedocs.io/en/stable/how-does-it-work.html That's the nice part about this kind of thing being opt-in: the definition time tooling *doesn't* have to cope with the full extent of Python's dynamic nature, as it can expect the user to be cooperating with the tool to some degree, and "you're not cooperating" can be a valid error to throw. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Steven D'Aprano writes:
Sometimes I think Python-as-a-teaching-language and Python-as-a- production-language are strongly opposed.
I haven't found that to be the case. Linters are quite effective, as long as you discipline the students to use them. I don't think this check is in the linters I use, and if not I'd like to see it added, though.
I wonder whether there might be a case to be made for a --with-training-wheels option?
You could make it, but I can't imagine any argument that doesn't immediately fall prey to
But that's better placed in the IDE, not the core language.
Maybe my imagination is just poor, but I think the burden of proof is on those who would add --with-training-wheels to the implementation. (N.B. This is an implementation issue, not something that would be in the language spec! :-)
Hello, While I think this should not be "on by default", I don't see the harm in being able to opt-in to this behavior. I also figured spending a few minutes attempting to write this would be fun: https://gist.github.com/veloutin/2ec3e5246651f5de78442516d8e24fc1 François: sorry about the double reply, I forgot to reply to the list. Vince
participants (14)
-
Adrián Orive Oneca -
Chris Barker -
fleblanc50 -
France3
-
François Leblanc
-
Guido van Rossum -
Nick Coghlan -
Nick Timkovich -
Paul Moore -
Sebastian Kreft -
Stephen J. Turnbull -
Steven D'Aprano -
Victor Stinner -
Vince Vinet