[Python-ideas] Binary arithmetic does not always call subclasses first
steve at pearwood.info
Mon Apr 24 14:25:06 EDT 2017
On Mon, Apr 24, 2017 at 05:57:17PM +1200, Greg Ewing wrote:
> Stephan Hoyer wrote:
> >In practice, CPython requires that the
> >right operand defines a different method before it defers to it.
> I'm not sure exactly what the rationale for this behaviour is,
> but it's probably something along the lines that the left
> method should already know how to deal with that combination
> of types, and right methods are only supposed to be called
> as a fallback if the left method can't handle the operands,
> so calling it in that situation would be wrong.
I've never seen that rationale before, and I don't think I would agree
with it. And it goes against the rationale in the docs:
[...] the right operand’s __rop__() method is tried before the
left operand’s __op__() method.
This is done so that a subclass can completely override binary
operators. Otherwise, the left operand’s __op__() method would
always accept the right operand: when an instance of a given
class is expected, an instance of a subclass of that class is
I think your rationale goes against the intention as documented. There's
no expectation that __rop__ methods are only to be called when the
__op__ method can't handle the operands.
In general, which operand "wins" should depend on the classes, not on
whether they happen to be on the left or right of the operator. (Except
in the case where we cannot decide between the operands, in which case
we break ties by preferring the __op__.) The reason is that subclasses
are usually intended to be more specialised than their parent, and so
they ought to be given priority in mixed operations.
Given classes X, Y(X), with instances x and y, we should expect that the
more specialised class (namely Y) gets called first whether we write:
x ⊕ y
y ⊕ x
for any operator ⊕.
As documented in the 3 docs, we get that for free: the interpreter
correctly calls the __op__ or __rop__ method, as needed, and the class
author doesn't have to think about it. That's how it's documented, but
not how it's implemented.
The alternative is that every class has to include boilerplate testing
for subclasses, as you say:
> Following that logic, the wrapper's __add__ method in your
> example needs to allow for the subclassing case, e.g.
> def __add__(self, other):
> t1 = type(self)
> t2 = type(other)
> t = t2 if issubclass(t2, t1) else t1
> return t(self.value + other.value)
but that's bad. That makes each and every class (that might ever be
subclassed) responsible for checking for subclasses, instead of putting
the check in one place (whichever part of the interpreter handles
calling __op__/__rop__ methods).
Remember that a specialised subclass might not overload the __op__ and
__rop__ methods themselves. It might overload a data attribute, or
another method that __op__ / __rop__ call.
def __add__(self, other):
__radd__ = __add__
A() + B()
As the more specialised instance (a subclass of A), the right hand
operand should get the priority.
> >the behavior is different for comparisons, which defer to
> >subclasses regardless of whether they implement a new method
> Comparisons are a bit different, because they don't have
> separate left and right methods, although it's hard to see
> exactly how that affects the logic.
It doesn't affect the logic, and comparisons implement exactly the
documented (in 3) behaviour. The only difference is that the
reversed methods aren't spelled __rop__:
__eq__ and __ne__ are their own reflection;
__lt__ and __gt__
__le__ and __ge__
py> class A(object):
... def __lt__(self, other):
... print("lt", self)
... return True
... def __gt__(self, other):
... print("gt", self)
... return False
py> class B(A):
py> A() < B()
gt <__main__.B object at 0xb7a9e8ec>
The more specialised class (B) has its method called, even though it
More information about the Python-ideas