On Sun, Apr 23, 2017 at 06:23:12PM -0700, Stephan Hoyer wrote:
I recently filed this as a bug, and was asked to repost to python-dev or python-ideas for greater visibility: http://bugs.python.org/issue30140
Without further ado, here is my original report: [...]
The reference documentation on binary arithmetic [3] states:
Note: If the right operand's type is a subclass of the left operand’s type and that subclass provides the reflected method for the operation, this method will be called before the left operand’s non-reflected method. This behavior allows subclasses to override their ancestors’ operations.
However, this isn't actually done if the right operand merely inherits from the left operand's type. In practice, CPython requires that the right operand defines a different method before it defers to it. Note that the behavior is different for comparisons, which defer to subclasses regardless of whether they implement a new method [4].
I think that's a bug. At the very least, comparisons and operators should behave the same. I think that the comparison behaviour is correct: the subclass reflected method should be called, even if the method isn't explicitly over-ridden.
I think this behavior is a mistake and should be corrected. It is just as useful to write generic binary arithmetic methods that are well defined on subclasses as generic comparison operations.
Agreed. [...]
Here is a simple example, of a well-behaved type that implements addition by wrapping its value and that returns NotImplemented when the other operand has the wrong type:
I often write classes like your example, and it is humbling to realise that they are buggy! I must admit I misread the docs and didn't notice the "and overrides" language, and clearly I didn't test my classes sufficiently or else I would have discovered this myself :-(
A does not defer to subclass B:
A(1) + B(1) A(2)
But it does defer to subclass C, which defines new methods (literally copied/pasted) for __add__/__radd__:
A(1) + C(1) C(2)
To me, that's the deciding point. We have two ways to go, and one of them encourages the copy-and-paste anti-pattern, the other doesn't. I think that's a Bad Thing and we should treat the comparison behaviour as correct, and the other operators are buggy. An explicit over-ride shouldn't be necessary.
Mark Dickinson's response:
The "Coercion rules" section[1] of the Python 2.7 docs is a bit more explicit about the intent:
""" Exception to the previous item: if the left operand is an instance of a built-in type or a new-style class, and the right operand is an instance of a proper subclass of that type or class and overrides the base’s __rop__() method, the right operand’s __rop__() method is tried before the left operand’s __op__() method. """
so the check for an override was clearly intentional, rather than an implementation convenience or accident. (It's also clearly intentional in the source and comments.)
The next paragraph tells us: This is done so that a subclass can completely override binary operators. Otherwise, the left operand’s __op__() method would always accept the right operand: when an instance of a given class is expected, an instance of a subclass of that class is always acceptable. but as far as I can tell, the comparison behaviour equally accomplishes that, without an explicit over-ride.
The 3.x docs don't have the "and overrides" language; I haven't figured out why and when that language changed.
[1] https://docs.python.org/release/2.7.6/reference/datamodel.html#coercion-rule...