[ python-Bugs-1153163 ] reflected operator not used when operands have the same type

[SourceForge.net]

Bugs item #1153163, was opened at 2005-02-28 01:09
Message generated for change (Settings changed) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1153163&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.4
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Hugh Secker-Walker (hughsw)
Assigned to: Nobody/Anonymous (nobody)
Summary: reflected operator not used when operands have the same type

Initial Comment:

The reflected operators, e.g. __radd__, are used when
the left operand does not have the non-reflected
operator, *unless* the right operand is of the same type.

The documentation on the "Emulating numeric types" page
doesn't mention this peculiar exclusion.  Of the
reflected operators it says:
"These methods are called to implement the binary
arithmetic operations (+, -, *, /, %, divmod(), pow(),
**, <<, >>, &, ^, |) with reflected (swapped) operands.
These functions are only called if the left operand
does not support the corresponding operation. For
instance, to evaluate the expression x-y, where y is an
instance of a class that has an __rsub__() method,
y.__rsub__(x) is called."

This code demonstrates the correct behavior and then
the problem:

class A(object):
    def __radd__(self, other):
        return '__radd__', other

print None + A()
print A() + A()

giving....

('__radd__', None)

Traceback (most recent call last):
  File "C:/Temp/reflectedbug.py", line 6, in -toplevel-
    print A() + A()
TypeError: unsupported operand type(s) for +: 'A' and 'A'

I've replaced None in the first print statement with
many kinds of builtin objects, instances of other
classes, and with instances of subclasses of A.  In all
these cases the __radd__ operator is used as
documented.  I've only seen the problem when the two
operands are of the same type.

This problem also occurs during the backing-off to
plain operators that occurs when augmented operators,
e.g. __iadd__, aren't implemented by the type and the
operands are of the same type.

This problem is present in 2.4 on Linux and Windows,
and in the current CVS version (2.5a0, 27-Feb-05) on Linux.


----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2006-06-14 08:32

Message:
Logged In: YES 
user_id=849994

I added some wording to the docs along your suggestions.
Fixed in rev. 46952, 46953.

----------------------------------------------------------------------

Comment By: Hugh Secker-Walker (hughsw)
Date: 2005-03-02 03:25

Message:
Logged In: YES 
user_id=1146279

Upon reflection and offline discussion it appears to me that
Python is behaving as designed and that the documentation is
buggy in that it fails to mention the special cases
demonstrated in this bug report.

The two special cases that the documentation should mention are:

1) The reflected operator is never used if the two operands
are of the same type, regardless of whether the
non-reflected operator exists.  

The necessity of this isn't clear to me.

2) If the type of the right-hand operand is a subclass of
the type of the left-hand operand, and if the right-hand
operand overloads the reflected operator from whatever is
(or isn't) implemented by the left-hand operand, then the
reflected operator of the right-hand operand will be used,
regardless of the presence of the non-reflected operator for
either type.  This behavior is necessary for subclasses to
overload parent class operators. 

-Hugh



----------------------------------------------------------------------

Comment By: Terry J. Reedy (tjreedy)
Date: 2005-03-01 05:08

Message:
Logged In: YES 
user_id=593130

I believe Python's designer(s) intend that if A()+A() is valid, then 
A will have an __add__ method.  __rxxx__ is only intended for 
backup use with mixed types.  So your example sends a mixed 
message to the interpreter.

If the current behavior is both intended and still necessary, the 
manual sentence
"These functions are only called if the left operand
does not support the corresponding operation"
could be augmented with 
"and has a different type than the right operand"


----------------------------------------------------------------------

Comment By: Hugh Secker-Walker (hughsw)
Date: 2005-03-01 04:36

Message:
Logged In: YES 
user_id=1146279

The problem is in the SLOT1BINFULL() macro near line 4020 in
typeobject.c.  In two places it ensures that the reflected
(reversed, swapped, rop<blah>, you name it) operator won't
be called if the two operands are of the same type. 
Removing these two exclusions fixes the problem.  

However, this being my third day ever modifying Python
source code, for all intents and purposes, I have no idea
why the exclusions were there (efficiency?).  And,
elsewhere, I saw high-level code that had a similar check on
operands having the same type with a comment that talked
about avoiding an infinite loop that could happen if there's
coercion and other subtly involved....

FWIW, with the changes I made, 256 tests are OK and 35 tests
are skipped -- as is usual on my Linux system.

I can post a trivial patch and figure out how to add a
regression test, but expert analysis is needed.

Also, the code in this macro (and perhaps helped by
abstract.c) implements curious and
not-documented-on-the-Emulating-numeric-types-page
semantics: if the operand on the right is a subtype of the
operand on the left and if  the right operand overloads the
reflected operator, then the reflected operator will be
called, even if the left-hand operand implements the regular
operator!  This is either a bug or it should be documented,
preferably with some rationale. E.g.

class A(object):
    def __add__(self, other):
        return 'A.__add__', other
class B(A):
    def __radd__(self, other):
        return 'B.__radd__', other

>>> B()+A()
('A.__add__', <__main__.A object at 0x00B65A30>)
>>> B()+B()
('A.__add__', <__main__.B object at 0x00B836F0>)
>>> 1+B()
('B.__radd__', 1)
>>> A()+B()
('B.__radd__', <__main__.A object at 0x00B65A30>)

Where the last one is what's curious or a bug.

-Hugh


----------------------------------------------------------------------

Comment By: Hugh Secker-Walker (hughsw)
Date: 2005-02-28 06:09

Message:
Logged In: YES 
user_id=1146279

I've looked into this a little.  Newbie that I am, I don't
know where the 
 x = slotw(v, w);
call goes (in binary_op1() in abstract.c near line 377)....
 AFAICT, this code in abstract.c behaves reasonably, so
problem would seem to be in the tp_as_number slot-function
that's getting called.  And whereever that is, it's not the
binary-op functions in classobject.c that I thought it would
be....


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1153163&group_id=5470