method decorators @final and @override in Python 2.4

Hi, If Python had method decorators @final (meaning: it is an error to override this method in any subclass) and @override (meaning: it is an error not having this method in a superclass), I would use them in my projects (some of them approaching 20 000 lines of Python code) and I'll feel more confident writing object-oriented Python code. Java already has similar decorators or specifiers. Do you think it is a good idea to have these in Python? I've created a proof-of-concept implementation, which uses metaclasses, and it works in Python 2.4 an Python 2.5. See http://www.math.bme.hu/~pts/pobjects.py and http://www.math.bme.hu/~pts/pobjects_example.py Best regards, Péter

Péter Szabó wrote:
+1 on the idea. however, using a metaclass would be to limiting imho. can you implement it in a different way? a lot of things people use metaclasses for work perfectly fine without them (instead use a superclass that overrides __new__ or similar). -apnzi

On Sat, Mar 28, 2009 at 10:21 AM, Mathias Panzenböck <grosser.meister.morti@gmx.net> wrote:
While it could be done by overriding __new__ in a superclass I'm not sure how that would make it easier to use, and it would make it harder to implement efficiently: this is a check that you would like to happen once at class definition time rather than on each instance creation. Of course you could do some caching to do it at the first instantiation only, but that still sounds clumsy; the metaclass is the obvious place to put this, and gives better error messages (at import instead of first use). But I don't think this idea is ripe for making it into a set of builtins yet, at least, I would prefer if someone coded this up as a 3rd party package and got feedback from a community of early adopters first. Or maybe one of the existing frameworks would be interested in adding this? While this may not be everyone's cup of tea (e.g. Scott David Daniels' reply), some frameworks cater to users who do like to be told when they're making this kind of errors. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Péter Szabó wrote:
I have no idea why you want these, and severe trepidation about dealing with code that uses them "just to be safe." It smacks of the over-use I see of doubled underscores. For @override, just because you've built a base class for one kind of object does not mean I have not thought of an interesting way to use 40% of your code to accomplish my own end. Why make me cut and paste? You are not responsible for the correctness of my flea-brained idea whether I inherit from your class or not. For @final, "how dare you" for similar reasons. Java at least has an excuse (compilation can proceed differently). --Scott David Daniels Scott.Daniels@Acm.Org

Scott David Daniels wrote:
I was asked off-group to give an example where use of @override prevents reusing some code. First, the above is an overstatement of my case, probably an attempt to "bully" you off that position. For that bullying, I apologize. Second, what follows below is one example of what @overrides prevents me from doing. Say you've built a class named "MostlyAbstract" with comparisons: class MostlyAbstract(object): @override def __hash__(self, other): pass @override def __lt__(self, other): pass @override def __eq__(self, other): pass def __le__(self, other): return self.__lt__(other) or self.__eq__(other) def __gt__(self, other): return other.__lt__(self) def __ge__(self, other): return self.__gt__(other) or self.__eq__(other) and I decide the comparison should works a bit differently: class MostAbstract(MostlyAbstract): def __gt__(self, other): return not self.__le__(self) This choice of mine won't work, even when I'm trying to just do a slight change to your abstraction. Similarly, If I want to monkey-path in a debugging print or two, I cannot do it without having to create a bunch of vacuous implementations. Also, a @final will prevent me from sneaking in aextra print when I'm bug-chasing. That being said, a mechanism like the following could be used as a facility to implement your two desires, by providing a nice simple place called as each class definition is completed: class Type(type): '''A MetaClass to call __initclass__ for freshly defined classes.''' def __new__(class_, name, supers, methods): if '__initclass__' in methods and not isinstance( methods['__initclass__'], classmethod): method = methods['__initclass__'] methods['__initclass__'] = classmethod(method) return type.__new__(class_, name, supers, methods) def __init__(self, name, supers, methods): type.__init__(self, name, supers, methods) if hasattr(self, '__initclass__'): self.__initclass__() In 2.5, for example, you'd use it like: class Foo(SomeParent): __metaclass__ = Type def __init_class__(self): <check for whatever you like.> --Scott David Daniels Scott.Daniels@Acm.Org

I think we have a different understanding what @override means. I define @override like this: ``class B(A): @override def F(self): pass'' is OK only if A.F is defined, i.e. there is a method F to override. What I understand about your mails is that your definition is: if there is @override on A.F, then any subclass of A must override A.F. Do I get the situation of the different understanding right? If so, do you find anything in my definition which prevents code reuse? (I don't.)
Thanks for the idea, this sounds generic enough for various uses, and it gives power to the author of the subclass. I'll see if my decorators can be implemented using __initclass__.

Péter Szabó wrote: ...
Nor do I. I completely misunderstood what you meant by override, and I agree that what you are specifying there _is_ a help to those writing code (I'd document it as a way of marking an intentional override). As to @final, I'd prefer a warning to an error when I override a final method. Overriding is a rich way of debugging, and if the point is to catch coding "misteaks", ignoring warnings is easier than changing package code when debugging. --Scott David Daniels Scott.Daniels@Acm.Org

On Mon, 30 Mar 2009 06:26:16 am Scott David Daniels wrote:
Perhaps I just haven't worked on enough 20,000 line projects, but I don't get the point of @override. It doesn't prevent somebody from writing (deliberately or accidentally) B.F in the absence of A.F, since the coder can simply leave off the @override. If @override is just a way of catching spelling mistakes, perhaps it would be better in pylint or pychecker. What have I missed? -- Steven D'Aprano

Steven D'Aprano wrote:
If @override is just a way of catching spelling mistakes, perhaps it would be better in pylint or pychecker. What have I missed?
If, for example, you have a huge testing framework, and some developers are given the task of developing elements from the framework by (say) overriding the test_sources and test_outcome methods, They can be handed an example module with @override demonstrating where to make the changes. class TestMondoDrive(DriveTestBase): @override def test_sources(self): return os.listdir('/standard/mondo/tests') @override def test_outcome(self, testname, outcome): if outcome != 'success': self.failures('At %s %s failed: %s' % ( time.strftime('%Y.%m.%d %H:%M:%S'), test_name, outcome)) else: assert False, "I've no idea how to deal with success" The resulting tests will be a bit easier to read, because you can easily distinguish between support methods and framework methods. Further, the entire warp drive test is not started if we stupidly spell the second "test_result" (as it was on the Enterprise tests). --Scott David Daniels Scott.Daniels@Acm.Org

On Mon, 30 Mar 2009 09:03:03 am Scott David Daniels wrote:
Classes B and A, method F. In case it is still unclear, I'm referencing Péter Szabó's post, which we both quoted:
The intention is that this will fail: class A: pass class B(A): @override def F(self): pass but this will be okay: class A: def F(self): pass class B(A): @override def F(self): pass But if I leave out the @override then I can define B.F regardless of whether or not A.F exists, so it doesn't prevent the creation of B.F.
"# OVERRIDE" or "# TODO" will do that just as well.
Maybe so, but comments and/or naming conventions do that too.
Further, the entire warp drive test is not started if we stupidly spell the second "test_result" (as it was on the Enterprise tests).
That's a reasonable benefit, but it still sounds to me like something that should go in pylint. I don't really have any objection to this, and Guido has already said it should go into a third party module first. Thank you for explaining the use-case. -- Steven D'Aprano

Steven D'Aprano wrote:
Yes, after all we did lose all of sector 4.66.73 on that unfortunate accident :-). I agree that it does feel a bit pylint-ish, but I have work on large unwieldy frameworks where large machines get powered on by the framework as part of running a test, and it is nice to see the whole test not even start in such circumstances. This is why I wrote that (easily ponied in) possible addition to type named "__initclass__", it seemed a more-useful technique that could be be used by the OP to implement his desires, while providing a simple place to put class initialization code that allows people to get a bit fancier with their classes without having to do the metaclass dance themselves. I'll try to putting up an ActiveState recipe for this in the coming week. --Scott David Daniels Scott.Daniels@Acm.Org

Just thinking... this sounds rather like trying to bolt interfaces into Python. In the 'consenting adults' view, shouldn't you be able to override a method that you inherit if you would like to? I can well imagine some well-meaning library author protecting some method with @final, then me spending hours cursing under my breath because I am unable to tweak the functionality in some new direction. If I understand what you are suggesting correctly, then I'm -1 on the idea. I would suggest that a good docstring could do the job just as well -- "Don't override this method in subclasses!". Do you have any use cases to highlight the problem you are trying to fix with this suggestion? Cheers, -T 2009/3/29 Péter Szabó <ptspts@gmail.com>
-- -------------------------------------------------- Tennessee Leeuwenburg http://myownhat.blogspot.com/ "Don't believe everything you think"

Péter Szabó <ptspts@gmail.com> writes:
If Python had method decorators @final (meaning: it is an error to override this method in any subclass)
What use case is there for this? It would have to be quite strong to override the Python philosophy that “we're all consenting adults here”, and that the programmer of the subclass is the one who knows best whether a method needs overriding.
and @override (meaning: it is an error not having this method in a superclass)
I'm not sure I understand this one, but if I'm right this is supported now with: class FooABC(object): def frobnicate(self): raise NotImplementedError("Must be implemented in derived class") Or perhaps: class FooABC(object): def __init__(self): if self.frobnicate is NotImplemented: raise ValueError("Must override 'frobnicate' in derived class") frobnicate = NotImplemented But, again, what is the use case? Is it strong enough to take away the ability of the derived class's implementor (who is, remember, a consenting adult) to take what they want from a class and leave the rest? -- \ “We can't depend for the long run on distinguishing one | `\ bitstream from another in order to figure out which rules | _o__) apply.” —Eben Moglen, _Anarchism Triumphant_, 1999 | Ben Finney

Ben Finney schrieb:
I agree. This goes in the same direction as suggesting private attributes. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.

Hi, Thanks for pointing out that a @final decorator in the superclass can be an obstacle for code reuse if we assume that the author of the subclass has no (or has only limited) control over the code of the superclass. I'll come up with features with which the subclass can bypass some or all decorators imposed in the superclass. (One easy way to do this right now is saying ``__metaclass__ = type'' in the subclass.) I agree that the programmer of the subclass is the one who knows best whether a method needs overriding. We have to give the programmer the power to enforce his will. But I think the errors coming from the decorators are very useful for notifying the programmer of the subclass that he is trying to something unexpected -- then he should make his decision to reconsider or enforce (e.g. @override_final as suggested by Nick Coghlan). By raising an error we inform the programmer that there is a decision he has to make. I think using a metaclass for implementing the checks based on the decorator is more appropriate than just overriding __new__ -- because we want to hook class creation (for which a metaclass is the adequate choice), not instance creation (for which overriding __new__ is the right choice). I definitely don't want any check at instance creation time, not even once. If I managed to create the class, it should be possible to create instances from it without decorator checks. By the way, as far as I can imaginge, using __new__ instead of the metaclass wouldn't make the implementations I can come up with simpler or shorter. A nice English docstring saying ``please don't override this method'' wouldn't make me happy. In my use case a few programmers including me are co-developing a fairly complex system in Python. There are tons of classes, tons of methods, each of them with docstrings. When I add some methods, I sometimes assume @final or @override, and I'm sure the system would break or function incorrectly if somebody added a subclass or changed a superclass ignoring my assumptions. Let's suppose this happens, but we don't notice it early enough; it becomes obvious only days or weeks later that the system cannot work this way, and the original reason of the problem was that somebody ignored a @final or @override assumption, because he didn't pay close attention to the thousands of docstrings. So we wast hours or days fixing the system. How can we avoid this problem in the future? Option A. Rely on writing and reading docstrings, everybody always correctly. Option B. Get an exception if a @final or @override assumption is violated. Option B is acceptable for me, Option A is not, because with option A there is no guarantee that the overlooking won't happen again. With Option B the programmer gets notified early, and he can reconsider his code or refactor my code early, must faster than fixing it weeks later. Best regards, Péter

Péter Szabó wrote:
Actually, that doesn't work as you might think...
The value assigned to '__metaclass__' (or the metaclass keyword argument in Py3k) is only one candidate metaclass that the metaclass determination algorithm considers - the metaclasses of all base classes are also candidates, and the algorithm picks the one which is a subclass of all of the candidate classes. If none of the candidates meet that criteria, then it complains loudly:
And remember, as far as @overrides goes, I believe @abc.abstractmethod already does what you want - it's only the @suggest_final/@override_final part of the idea that doesn't exist. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

Ben Finney wrote:
Agreed - the base class author has no right to tell subclass authors that they *can't* do something. They can give hints that something shouldn't be messed with by using a leading underscore and leaving it undocumented (or advising against overriding it in the documentation). That said, if a @suggest_final decorator was paired with an @override_final decorator, I could actually see the point: one thing that can happen with undocumented private methods and attributes in a large class heirarchy is a subclass *accidentally* overriding them, which can then lead to bugs which are tricky to track down (avoiding such conflicts is actually one of the legitimate use cases for name mangling). A suggest_final/override_final decorator pair would flag accidental naming conflicts in complicated heirarchies at class definition time, while still granting the subclass author the ability to replace the nominally 'final' methods if they found it necessary.
Even better:
Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

On Sat, Mar 28, 2009 at 7:40 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I'm sorry, but this is going too far. There are plenty of situations where, indeed, this ought to be only a hint, but I think it goes to far to say that a base class can never have the last word about something. Please note that I already suggested this be put in a 3rd party package -- I'm not about to make these builtins. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
Sorry, what I wrote was broader in scope than what I actually meant. I only intended to refer to otherwise arbitrary non-functional constraints like marking elements of the base as "private" or "final" without giving a subclass author a way to override them (after all, even name mangling can be reversed with sufficient motivation). A base class obviously needs to impose some real constraints on subclasses in practice, or it isn't going to be a very useful (if nothing else, it needs to set down the details of the shared API). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

On Sat, Mar 28, 2009 at 10:14 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
To paraphrase a cliche: "Having 'private' (or 'final') in a language doesn't cause unusable software. People using 'private' (or 'final') indiscriminately cause unusable software." :-)
-- --Guido van Rossum (home page: http://www.python.org/~guido/)

Péter Szabó <ptspts@gmail.com> wrote:
No on @final (I've had more trouble with ill-considered Java "final" classes than I can believe), but @override sounds interesting. I can see the point of that. Should do the check at compile time, right? Bill

Péter Szabó wrote:
+1 on the idea. however, using a metaclass would be to limiting imho. can you implement it in a different way? a lot of things people use metaclasses for work perfectly fine without them (instead use a superclass that overrides __new__ or similar). -apnzi

On Sat, Mar 28, 2009 at 10:21 AM, Mathias Panzenböck <grosser.meister.morti@gmx.net> wrote:
While it could be done by overriding __new__ in a superclass I'm not sure how that would make it easier to use, and it would make it harder to implement efficiently: this is a check that you would like to happen once at class definition time rather than on each instance creation. Of course you could do some caching to do it at the first instantiation only, but that still sounds clumsy; the metaclass is the obvious place to put this, and gives better error messages (at import instead of first use). But I don't think this idea is ripe for making it into a set of builtins yet, at least, I would prefer if someone coded this up as a 3rd party package and got feedback from a community of early adopters first. Or maybe one of the existing frameworks would be interested in adding this? While this may not be everyone's cup of tea (e.g. Scott David Daniels' reply), some frameworks cater to users who do like to be told when they're making this kind of errors. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Péter Szabó wrote:
I have no idea why you want these, and severe trepidation about dealing with code that uses them "just to be safe." It smacks of the over-use I see of doubled underscores. For @override, just because you've built a base class for one kind of object does not mean I have not thought of an interesting way to use 40% of your code to accomplish my own end. Why make me cut and paste? You are not responsible for the correctness of my flea-brained idea whether I inherit from your class or not. For @final, "how dare you" for similar reasons. Java at least has an excuse (compilation can proceed differently). --Scott David Daniels Scott.Daniels@Acm.Org

Scott David Daniels wrote:
I was asked off-group to give an example where use of @override prevents reusing some code. First, the above is an overstatement of my case, probably an attempt to "bully" you off that position. For that bullying, I apologize. Second, what follows below is one example of what @overrides prevents me from doing. Say you've built a class named "MostlyAbstract" with comparisons: class MostlyAbstract(object): @override def __hash__(self, other): pass @override def __lt__(self, other): pass @override def __eq__(self, other): pass def __le__(self, other): return self.__lt__(other) or self.__eq__(other) def __gt__(self, other): return other.__lt__(self) def __ge__(self, other): return self.__gt__(other) or self.__eq__(other) and I decide the comparison should works a bit differently: class MostAbstract(MostlyAbstract): def __gt__(self, other): return not self.__le__(self) This choice of mine won't work, even when I'm trying to just do a slight change to your abstraction. Similarly, If I want to monkey-path in a debugging print or two, I cannot do it without having to create a bunch of vacuous implementations. Also, a @final will prevent me from sneaking in aextra print when I'm bug-chasing. That being said, a mechanism like the following could be used as a facility to implement your two desires, by providing a nice simple place called as each class definition is completed: class Type(type): '''A MetaClass to call __initclass__ for freshly defined classes.''' def __new__(class_, name, supers, methods): if '__initclass__' in methods and not isinstance( methods['__initclass__'], classmethod): method = methods['__initclass__'] methods['__initclass__'] = classmethod(method) return type.__new__(class_, name, supers, methods) def __init__(self, name, supers, methods): type.__init__(self, name, supers, methods) if hasattr(self, '__initclass__'): self.__initclass__() In 2.5, for example, you'd use it like: class Foo(SomeParent): __metaclass__ = Type def __init_class__(self): <check for whatever you like.> --Scott David Daniels Scott.Daniels@Acm.Org

I think we have a different understanding what @override means. I define @override like this: ``class B(A): @override def F(self): pass'' is OK only if A.F is defined, i.e. there is a method F to override. What I understand about your mails is that your definition is: if there is @override on A.F, then any subclass of A must override A.F. Do I get the situation of the different understanding right? If so, do you find anything in my definition which prevents code reuse? (I don't.)
Thanks for the idea, this sounds generic enough for various uses, and it gives power to the author of the subclass. I'll see if my decorators can be implemented using __initclass__.

Péter Szabó wrote: ...
Nor do I. I completely misunderstood what you meant by override, and I agree that what you are specifying there _is_ a help to those writing code (I'd document it as a way of marking an intentional override). As to @final, I'd prefer a warning to an error when I override a final method. Overriding is a rich way of debugging, and if the point is to catch coding "misteaks", ignoring warnings is easier than changing package code when debugging. --Scott David Daniels Scott.Daniels@Acm.Org

On Mon, 30 Mar 2009 06:26:16 am Scott David Daniels wrote:
Perhaps I just haven't worked on enough 20,000 line projects, but I don't get the point of @override. It doesn't prevent somebody from writing (deliberately or accidentally) B.F in the absence of A.F, since the coder can simply leave off the @override. If @override is just a way of catching spelling mistakes, perhaps it would be better in pylint or pychecker. What have I missed? -- Steven D'Aprano

Steven D'Aprano wrote:
If @override is just a way of catching spelling mistakes, perhaps it would be better in pylint or pychecker. What have I missed?
If, for example, you have a huge testing framework, and some developers are given the task of developing elements from the framework by (say) overriding the test_sources and test_outcome methods, They can be handed an example module with @override demonstrating where to make the changes. class TestMondoDrive(DriveTestBase): @override def test_sources(self): return os.listdir('/standard/mondo/tests') @override def test_outcome(self, testname, outcome): if outcome != 'success': self.failures('At %s %s failed: %s' % ( time.strftime('%Y.%m.%d %H:%M:%S'), test_name, outcome)) else: assert False, "I've no idea how to deal with success" The resulting tests will be a bit easier to read, because you can easily distinguish between support methods and framework methods. Further, the entire warp drive test is not started if we stupidly spell the second "test_result" (as it was on the Enterprise tests). --Scott David Daniels Scott.Daniels@Acm.Org

On Mon, 30 Mar 2009 09:03:03 am Scott David Daniels wrote:
Classes B and A, method F. In case it is still unclear, I'm referencing Péter Szabó's post, which we both quoted:
The intention is that this will fail: class A: pass class B(A): @override def F(self): pass but this will be okay: class A: def F(self): pass class B(A): @override def F(self): pass But if I leave out the @override then I can define B.F regardless of whether or not A.F exists, so it doesn't prevent the creation of B.F.
"# OVERRIDE" or "# TODO" will do that just as well.
Maybe so, but comments and/or naming conventions do that too.
Further, the entire warp drive test is not started if we stupidly spell the second "test_result" (as it was on the Enterprise tests).
That's a reasonable benefit, but it still sounds to me like something that should go in pylint. I don't really have any objection to this, and Guido has already said it should go into a third party module first. Thank you for explaining the use-case. -- Steven D'Aprano

Steven D'Aprano wrote:
Yes, after all we did lose all of sector 4.66.73 on that unfortunate accident :-). I agree that it does feel a bit pylint-ish, but I have work on large unwieldy frameworks where large machines get powered on by the framework as part of running a test, and it is nice to see the whole test not even start in such circumstances. This is why I wrote that (easily ponied in) possible addition to type named "__initclass__", it seemed a more-useful technique that could be be used by the OP to implement his desires, while providing a simple place to put class initialization code that allows people to get a bit fancier with their classes without having to do the metaclass dance themselves. I'll try to putting up an ActiveState recipe for this in the coming week. --Scott David Daniels Scott.Daniels@Acm.Org

Just thinking... this sounds rather like trying to bolt interfaces into Python. In the 'consenting adults' view, shouldn't you be able to override a method that you inherit if you would like to? I can well imagine some well-meaning library author protecting some method with @final, then me spending hours cursing under my breath because I am unable to tweak the functionality in some new direction. If I understand what you are suggesting correctly, then I'm -1 on the idea. I would suggest that a good docstring could do the job just as well -- "Don't override this method in subclasses!". Do you have any use cases to highlight the problem you are trying to fix with this suggestion? Cheers, -T 2009/3/29 Péter Szabó <ptspts@gmail.com>
-- -------------------------------------------------- Tennessee Leeuwenburg http://myownhat.blogspot.com/ "Don't believe everything you think"

Péter Szabó <ptspts@gmail.com> writes:
If Python had method decorators @final (meaning: it is an error to override this method in any subclass)
What use case is there for this? It would have to be quite strong to override the Python philosophy that “we're all consenting adults here”, and that the programmer of the subclass is the one who knows best whether a method needs overriding.
and @override (meaning: it is an error not having this method in a superclass)
I'm not sure I understand this one, but if I'm right this is supported now with: class FooABC(object): def frobnicate(self): raise NotImplementedError("Must be implemented in derived class") Or perhaps: class FooABC(object): def __init__(self): if self.frobnicate is NotImplemented: raise ValueError("Must override 'frobnicate' in derived class") frobnicate = NotImplemented But, again, what is the use case? Is it strong enough to take away the ability of the derived class's implementor (who is, remember, a consenting adult) to take what they want from a class and leave the rest? -- \ “We can't depend for the long run on distinguishing one | `\ bitstream from another in order to figure out which rules | _o__) apply.” —Eben Moglen, _Anarchism Triumphant_, 1999 | Ben Finney

Ben Finney schrieb:
I agree. This goes in the same direction as suggesting private attributes. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.

Hi, Thanks for pointing out that a @final decorator in the superclass can be an obstacle for code reuse if we assume that the author of the subclass has no (or has only limited) control over the code of the superclass. I'll come up with features with which the subclass can bypass some or all decorators imposed in the superclass. (One easy way to do this right now is saying ``__metaclass__ = type'' in the subclass.) I agree that the programmer of the subclass is the one who knows best whether a method needs overriding. We have to give the programmer the power to enforce his will. But I think the errors coming from the decorators are very useful for notifying the programmer of the subclass that he is trying to something unexpected -- then he should make his decision to reconsider or enforce (e.g. @override_final as suggested by Nick Coghlan). By raising an error we inform the programmer that there is a decision he has to make. I think using a metaclass for implementing the checks based on the decorator is more appropriate than just overriding __new__ -- because we want to hook class creation (for which a metaclass is the adequate choice), not instance creation (for which overriding __new__ is the right choice). I definitely don't want any check at instance creation time, not even once. If I managed to create the class, it should be possible to create instances from it without decorator checks. By the way, as far as I can imaginge, using __new__ instead of the metaclass wouldn't make the implementations I can come up with simpler or shorter. A nice English docstring saying ``please don't override this method'' wouldn't make me happy. In my use case a few programmers including me are co-developing a fairly complex system in Python. There are tons of classes, tons of methods, each of them with docstrings. When I add some methods, I sometimes assume @final or @override, and I'm sure the system would break or function incorrectly if somebody added a subclass or changed a superclass ignoring my assumptions. Let's suppose this happens, but we don't notice it early enough; it becomes obvious only days or weeks later that the system cannot work this way, and the original reason of the problem was that somebody ignored a @final or @override assumption, because he didn't pay close attention to the thousands of docstrings. So we wast hours or days fixing the system. How can we avoid this problem in the future? Option A. Rely on writing and reading docstrings, everybody always correctly. Option B. Get an exception if a @final or @override assumption is violated. Option B is acceptable for me, Option A is not, because with option A there is no guarantee that the overlooking won't happen again. With Option B the programmer gets notified early, and he can reconsider his code or refactor my code early, must faster than fixing it weeks later. Best regards, Péter

Péter Szabó wrote:
Actually, that doesn't work as you might think...
The value assigned to '__metaclass__' (or the metaclass keyword argument in Py3k) is only one candidate metaclass that the metaclass determination algorithm considers - the metaclasses of all base classes are also candidates, and the algorithm picks the one which is a subclass of all of the candidate classes. If none of the candidates meet that criteria, then it complains loudly:
And remember, as far as @overrides goes, I believe @abc.abstractmethod already does what you want - it's only the @suggest_final/@override_final part of the idea that doesn't exist. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

Ben Finney wrote:
Agreed - the base class author has no right to tell subclass authors that they *can't* do something. They can give hints that something shouldn't be messed with by using a leading underscore and leaving it undocumented (or advising against overriding it in the documentation). That said, if a @suggest_final decorator was paired with an @override_final decorator, I could actually see the point: one thing that can happen with undocumented private methods and attributes in a large class heirarchy is a subclass *accidentally* overriding them, which can then lead to bugs which are tricky to track down (avoiding such conflicts is actually one of the legitimate use cases for name mangling). A suggest_final/override_final decorator pair would flag accidental naming conflicts in complicated heirarchies at class definition time, while still granting the subclass author the ability to replace the nominally 'final' methods if they found it necessary.
Even better:
Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

On Sat, Mar 28, 2009 at 7:40 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I'm sorry, but this is going too far. There are plenty of situations where, indeed, this ought to be only a hint, but I think it goes to far to say that a base class can never have the last word about something. Please note that I already suggested this be put in a 3rd party package -- I'm not about to make these builtins. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
Sorry, what I wrote was broader in scope than what I actually meant. I only intended to refer to otherwise arbitrary non-functional constraints like marking elements of the base as "private" or "final" without giving a subclass author a way to override them (after all, even name mangling can be reversed with sufficient motivation). A base class obviously needs to impose some real constraints on subclasses in practice, or it isn't going to be a very useful (if nothing else, it needs to set down the details of the shared API). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

On Sat, Mar 28, 2009 at 10:14 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
To paraphrase a cliche: "Having 'private' (or 'final') in a language doesn't cause unusable software. People using 'private' (or 'final') indiscriminately cause unusable software." :-)
-- --Guido van Rossum (home page: http://www.python.org/~guido/)

Péter Szabó <ptspts@gmail.com> wrote:
No on @final (I've had more trouble with ill-considered Java "final" classes than I can believe), but @override sounds interesting. I can see the point of that. Should do the check at compile time, right? Bill
participants (10)
-
Ben Finney
-
Bill Janssen
-
Georg Brandl
-
Guido van Rossum
-
Mathias Panzenböck
-
Nick Coghlan
-
Péter Szabó
-
Scott David Daniels
-
Steven D'Aprano
-
Tennessee Leeuwenburg