[Python-Dev] The role of NotImplemented: What is it for and when should it be used?

[Ethan Furman]

Just to be clear, this is about NotImplemented, not NotImplementedError.

tl;dr  When a binary operation fails, should an exception be raised or NotImplemented returned?


When a binary operation in Python is attempted, there are two possibilities:

   - it can work
   - it can't work

The main reason [1] that it can't work is that the two operands are of different types, and the first type does not know 
how to deal with the second type.

The question then becomes: how does the first type tell Python that it cannot perform the requested operation?  The most 
obvious answer is to raise an exception, and TypeError is a good candidate.  The problem with the exception raising 
approach is that once an exception is raised, Python doesn't try anything else to make the operation work.

What's wrong with that?  Well, the second type might know how to perform the operation, and in fact that is why we have 
the reflected special methods, such as __radd__ and __rmod__ -- but if the first type raises an exception the __rxxx__ 
methods will not be tried.

Okay, how can the first type tell Python that it cannot do what is requested, but to go ahead and check with the second 
type to see if it does?  That is where NotImplemented comes in -- if a special method (and only a special method) 
returns NotImplemented then Python will check to see if there is anything else it can do to make the operation succeed; 
if all attempts return NotImplemented, then Python itself will raise an appropriate exception [2].

In an effort to see how often NotImplemented is currently being returned I crafted a test script [3] to test the types 
bytes, bytearray, str, dict, list, tuple, Enum, Counter, defaultdict, deque, and OrderedDict with the operations for 
__add__, __and__, __floordiv__, __iadd__, __iand__, __ifloordiv__, __ilshift__, __imod__, __imul__, __ior__, __ipow__, 
__irshift__, __isub__, __itruediv__, __ixor__, __lshift__, __mod__, __mul__, __or__, __pow__, __rshift__, __sub__, 
__truediv__, and __xor__.

Here are the results of the 275 tests:
--------------------------------------------------------------------------------
testing control...

ipow -- Exception <unsupported operand type(s) for ** or pow(): 'Control' and 'subtype'> raised
errors in Control -- misunderstanding or bug?

testing types against a foreign class

iadd(Counter()) -- Exception <'SomeOtherClass' object has no attribute 'items'> raised instead of TypeError
iand(Counter()) -- NotImplemented not returned, TypeError not raised
ior(Counter()) -- Exception <'SomeOtherClass' object has no attribute 'items'> raised instead of TypeError
isub(Counter()) -- Exception <'SomeOtherClass' object has no attribute 'items'> raised instead of TypeError


testing types against a subclass

mod(str()) -- NotImplemented not returned, TypeError not raised

iadd(Counter()) -- Exception <'subtype' object has no attribute 'items'> raised (should have worked)
iand(Counter()) -- NotImplemented not returned, TypeError not raised
ior(Counter()) -- Exception <'subtype' object has no attribute 'items'> raised (should have worked)
isub(Counter()) -- Exception <'subtype' object has no attribute 'items'> raised (should have worked)
--------------------------------------------------------------------------------

Two observations:

   - __ipow__ doesn't seem to behave properly in the 3.x line (that error doesn't show up when testing against 2.7)

   - Counter should be returning NotImplemented instead of raising an AttributeError, for three reasons [4]:
     - a TypeError is more appropriate
     - subclasses /cannot/ work with the current implementation
     - __iand__ is currently a silent failure if the Counter is empty, and the other operand should trigger a failure

Back to the main point...

So, if my understanding is correct:

   - NotImplemented is used to signal Python that the requested operation could not be performed
   - it should be used by the binary special methods to signal type mismatch failure, so any subclass gets a chance to work.

Is my understanding correct?  Is this already in the docs somewhere, and I just missed it?

--
~Ethan~

[1] at least, it's the main reason in my code
[2] usually a TypeError, stating either that the operation is not supported, or the types are unorderable
[3] test script at the end
[4] https://bugs.python.org/issue22766 [returning NotImplemented was rejected]

-- 8< ----------------------------------------------------------------------------
from collections import Counter, defaultdict, deque, OrderedDict
from fractions import Fraction
from decimal import Decimal
from enum import Enum
import operator
import sys

py_ver = sys.version_info[:2]

types = (
     bytes, bytearray, str, dict, list, tuple,
     Enum, Counter, defaultdict, deque, OrderedDict,
     )
numeric_types = int, float, Decimal, Fraction

operators = (
     '__add__', '__and__', '__floordiv__',
     '__iadd__', '__iand__', '__ifloordiv__', '__ilshift__',
     '__imod__', '__imul__', '__ior__', '__ipow__',
     '__irshift__', '__isub__', '__itruediv__', '__ixor__',
     '__lshift__', '__mod__', '__mul__',
     '__or__', '__pow__', '__rshift__', '__sub__', '__truediv__',
     '__xor__',
     )

if py_ver >= (3, 0):
     operators += ('__gt__',  '__ge__', '__le__','__lt__')

ordered_reflections = {
         '__le__': '__ge__',
         '__lt__': '__gt__',
         '__ge__': '__le__',
         '__gt__': '__lt__',
         }


# helpers

class SomeOtherClass:
     """"
     used to test behavior when a different type is passed in to the
     special methods
     """
     def __repr__(self):
         return 'SomeOtherClass'
some_other_class = SomeOtherClass()

class MainClassHandled(Exception):
     """
     called by base class if both operands are of type base class
     """

class SubClassCalled(Exception):
     """
     called by reflected operations for testing
     """

def create_control(test_op):
     def _any(self, other):
         if not type(other) is self.__class__:
             return NotImplemented
         raise MainClassHandled
     class Control:
         "returns NotImplemented when other object is not supported"
     _any.__name__ = op
     setattr(Control, test_op, _any)
     return Control()

def create_subtype(test_op, base_class=object):
     def _any(*a):
         global subclass_called
         subclass_called = True
         raise SubClassCalled
     class subtype(base_class):
         __add__ = __sub__ = __mul__ = __truediv__ = __floordiv__ = _any
         __mod__ = __divmod__ = __pow__ = __lshift__ = __rshift__ = _any
         __and__ = __xor__ = __or__ = _any
         __radd__ = __rsub__ = __rmul__ = __rtruediv__ = __rfloordiv__ = _any
         __rmod__ = __rdivmod__ = __rpow__ = __rlshift__ = __rrshift__ = _any
         __rand__ = __rxor__ = __ror__ = _any
         __le__ = __lt__ = __gt__ = __ge__ = _any
     if issubclass(subtype, (bytes, bytearray)):
         value = b'hello'
     elif issubclass(subtype, str):
         value = 'goodbye'
     elif issubclass(subtype, (list, tuple)):
         value = (1, 2, 3)
     elif issubclass(subtype, (int, float, Decimal, Fraction)):
         value = 42
     else:
         # ignore value
         return subtype()
     return subtype(value)


# test exceptions

# control against some other class
print('testing control...\n')
errors = False
for op in operators:
     control = create_control(op)
     op = getattr(operator, op)
     try:
         op(control, some_other_class)
     except TypeError:
         # the end result of no method existing, or each method called returning
         # NotImplemented because it does not know how to perform the requested
         # operation between the two types
         pass
     except Exception as exc:
         errors = True
         print('%s(%s()) -- Exception <%s> raised instead of TypeError' %
                 (op.__name__, test_type.__name__, exc))
     else:
         errors = True
         print('Control -- TypeError not raised for op %r' % op)
if errors:
     print('errors in Control -- misunderstanding or bug?\n')

# control against a subclass
errors = False
for op in operators:
     subclass_called = False
     control = create_control(op)
     subtype = create_subtype(op, control.__class__)
     op = getattr(operator, op)
     try:
         op(control, subtype)
     except SubClassCalled:
         # if the control class properly signals that it doesn't know how to
         # perform the operation, of if Python notices that a reflected
         # operation exists, we get here (which is good)
         pass
     except MainClassHandled:
         errors = True
         print('Control did not yield to subclass for op %r' % op)
     except Exception as exc:
         if subclass_called:
             # exception was subverted to something more appropriate (like
             # unorderable types)
             pass
         errors = True
         print('%s -- Exception <%s> raised' %
                 (op.__name__, exc))
     else:
         errors = True
         print('Control -- op %r appears to have succeeded (it should not have)' % op)
if errors:
     print('errors in Control -- misunderstanding or bug?\n')


# tests
print('testing types against a foreign class\n')
for test_type in types + numeric_types:
     errors = False
     for op in operators:
         op = getattr(operator, op)
         try:
             op(test_type(), some_other_class)
         except TypeError:
             pass
         except Exception as exc:
             errors = True
             print('%s(%s()) -- Exception <%s> raised instead of TypeError' %
                     (op.__name__, test_type.__name__, exc))
         else:
             print('%s(%s()) -- NotImplemented not returned, TypeError not raised' %
                    (op.__name__, test_type.__name__))
     if errors:
         print()

print()

# test subclasses
print('testing types against a subclass\n')
for test_type in types:
     errors = False
     for op in operators:
         subclass_called = False
         if not test_type.__dict__.get(op):
             continue
         subclass = create_subtype(op, test_type)
         op = getattr(operator, op)
         try:
             if test_type is str:
                 op('%s', subtype)
             else:
                 op(test_type(), subtype)
         except SubClassCalled:
             # expected, ignore
             pass
         except Exception as exc:
             if subclass_called:
                 # exception raised by subclass was changed
                 pass
             errors = True
             print('%s(%s()) -- Exception <%s> raised (should have worked)' %
                     (op.__name__, test_type.__name__, exc))
         else:
             errors = True
             print('%s(%s()) -- NotImplemented not returned, TypeError not raised' %
                     (op.__name__, test_type.__name__))
     if errors:
         print()
for test_type in numeric_types:
     errors = False
     for op in operators:
         subclass_called = False
         if not test_type.__dict__.get(op):
             continue
         subtype = create_subtype(op, test_type)
         op = getattr(operator, op)
         try:
             op(test_type(), subtype)
         except SubClassCalled:
             # expected, ignore
             pass
         except Exception as exc:
             if subclass_called:
                 # exception raised by subclass was changed
                 pass
             errors = True
             print('%s(%s()) -- Exception <%s> raised (should have worked)' %
                     (op.__name__, test_type.__name__, exc))
         else:
             errors = True
             print('%s(%s)) -- NotImplemented not returned' %
                     (op.__name__, test_type.__name__))
     if errors:
         print()
-- 8< ----------------------------------------------------------------------------