<p dir="ltr">Not those.</p>
<div class="gmail_quote">On Nov 3, 2014 8:56 AM, "Antoine Pitrou" <<a href="mailto:solipsis@pitrou.net">solipsis@pitrou.net</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Mon, 3 Nov 2014 08:48:07 -0800<br>
Guido van Rossum <<a href="mailto:guido@python.org">guido@python.org</a>> wrote:<br>
> Gotta be brief, but NotImplemented is for all binary ops.<br>
<br>
Even in-place ops?<br>
<br>
Regards<br>
<br>
Antoine.<br>
<br>
<br>
> Power may be an<br>
> exception because it's ternary?<br>
> On Nov 3, 2014 8:08 AM, "Brett Cannon" <<a href="mailto:brett@python.org">brett@python.org</a>> wrote:<br>
><br>
> ><br>
> ><br>
> > On Mon Nov 03 2014 at 5:31:21 AM Ethan Furman <<a href="mailto:ethan@stoneleaf.us">ethan@stoneleaf.us</a>> wrote:<br>
> ><br>
> >> Just to be clear, this is about NotImplemented, not NotImplementedError.<br>
> >><br>
> >> tl;dr When a binary operation fails, should an exception be raised or<br>
> >> NotImplemented returned?<br>
> >><br>
> ><br>
> > The docs for NotImplemented suggest it's only for rich comparison methods<br>
> > and not all binary operators:<br>
> > <a href="https://docs.python.org/3/library/constants.html#NotImplemented" target="_blank">https://docs.python.org/3/library/constants.html#NotImplemented</a> . But<br>
> > then had I not read that I would have said all binary operator methods<br>
> > should return NotImplemented when the types are incompatible.<br>
> ><br>
> > -Brett<br>
> ><br>
> ><br>
> >><br>
> >><br>
> >> When a binary operation in Python is attempted, there are two<br>
> >> possibilities:<br>
> >><br>
> >> - it can work<br>
> >> - it can't work<br>
> >><br>
> >> The main reason [1] that it can't work is that the two operands are of<br>
> >> different types, and the first type does not know<br>
> >> how to deal with the second type.<br>
> >><br>
> >> The question then becomes: how does the first type tell Python that it<br>
> >> cannot perform the requested operation? The most<br>
> >> obvious answer is to raise an exception, and TypeError is a good<br>
> >> candidate. The problem with the exception raising<br>
> >> approach is that once an exception is raised, Python doesn't try anything<br>
> >> else to make the operation work.<br>
> >><br>
> >> What's wrong with that? Well, the second type might know how to perform<br>
> >> the operation, and in fact that is why we have<br>
> >> the reflected special methods, such as __radd__ and __rmod__ -- but if<br>
> >> the first type raises an exception the __rxxx__<br>
> >> methods will not be tried.<br>
> >><br>
> >> Okay, how can the first type tell Python that it cannot do what is<br>
> >> requested, but to go ahead and check with the second<br>
> >> type to see if it does? That is where NotImplemented comes in -- if a<br>
> >> special method (and only a special method)<br>
> >> returns NotImplemented then Python will check to see if there is anything<br>
> >> else it can do to make the operation succeed;<br>
> >> if all attempts return NotImplemented, then Python itself will raise an<br>
> >> appropriate exception [2].<br>
> >><br>
> >> In an effort to see how often NotImplemented is currently being returned<br>
> >> I crafted a test script [3] to test the types<br>
> >> bytes, bytearray, str, dict, list, tuple, Enum, Counter, defaultdict,<br>
> >> deque, and OrderedDict with the operations for<br>
> >> __add__, __and__, __floordiv__, __iadd__, __iand__, __ifloordiv__,<br>
> >> __ilshift__, __imod__, __imul__, __ior__, __ipow__,<br>
> >> __irshift__, __isub__, __itruediv__, __ixor__, __lshift__, __mod__,<br>
> >> __mul__, __or__, __pow__, __rshift__, __sub__,<br>
> >> __truediv__, and __xor__.<br>
> >><br>
> >> Here are the results of the 275 tests:<br>
> >> ------------------------------------------------------------<br>
> >> --------------------<br>
> >> testing control...<br>
> >><br>
> >> ipow -- Exception <unsupported operand type(s) for ** or pow(): 'Control'<br>
> >> and 'subtype'> raised<br>
> >> errors in Control -- misunderstanding or bug?<br>
> >><br>
> >> testing types against a foreign class<br>
> >><br>
> >> iadd(Counter()) -- Exception <'SomeOtherClass' object has no attribute<br>
> >> 'items'> raised instead of TypeError<br>
> >> iand(Counter()) -- NotImplemented not returned, TypeError not raised<br>
> >> ior(Counter()) -- Exception <'SomeOtherClass' object has no attribute<br>
> >> 'items'> raised instead of TypeError<br>
> >> isub(Counter()) -- Exception <'SomeOtherClass' object has no attribute<br>
> >> 'items'> raised instead of TypeError<br>
> >><br>
> >><br>
> >> testing types against a subclass<br>
> >><br>
> >> mod(str()) -- NotImplemented not returned, TypeError not raised<br>
> >><br>
> >> iadd(Counter()) -- Exception <'subtype' object has no attribute 'items'><br>
> >> raised (should have worked)<br>
> >> iand(Counter()) -- NotImplemented not returned, TypeError not raised<br>
> >> ior(Counter()) -- Exception <'subtype' object has no attribute 'items'><br>
> >> raised (should have worked)<br>
> >> isub(Counter()) -- Exception <'subtype' object has no attribute 'items'><br>
> >> raised (should have worked)<br>
> >> ------------------------------------------------------------<br>
> >> --------------------<br>
> >><br>
> >> Two observations:<br>
> >><br>
> >> - __ipow__ doesn't seem to behave properly in the 3.x line (that error<br>
> >> doesn't show up when testing against 2.7)<br>
> >><br>
> >> - Counter should be returning NotImplemented instead of raising an<br>
> >> AttributeError, for three reasons [4]:<br>
> >> - a TypeError is more appropriate<br>
> >> - subclasses /cannot/ work with the current implementation<br>
> >> - __iand__ is currently a silent failure if the Counter is empty,<br>
> >> and the other operand should trigger a failure<br>
> >><br>
> >> Back to the main point...<br>
> >><br>
> >> So, if my understanding is correct:<br>
> >><br>
> >> - NotImplemented is used to signal Python that the requested operation<br>
> >> could not be performed<br>
> >> - it should be used by the binary special methods to signal type<br>
> >> mismatch failure, so any subclass gets a chance to work.<br>
> >><br>
> >> Is my understanding correct? Is this already in the docs somewhere, and<br>
> >> I just missed it?<br>
> >><br>
> >> --<br>
> >> ~Ethan~<br>
> >><br>
> >> [1] at least, it's the main reason in my code<br>
> >> [2] usually a TypeError, stating either that the operation is not<br>
> >> supported, or the types are unorderable<br>
> >> [3] test script at the end<br>
> >> [4] <a href="https://bugs.python.org/issue22766" target="_blank">https://bugs.python.org/issue22766</a> [returning NotImplemented was<br>
> >> rejected]<br>
> >><br>
> >> -- 8< ------------------------------------------------------------<br>
> >> ----------------<br>
> >> from collections import Counter, defaultdict, deque, OrderedDict<br>
> >> from fractions import Fraction<br>
> >> from decimal import Decimal<br>
> >> from enum import Enum<br>
> >> import operator<br>
> >> import sys<br>
> >><br>
> >> py_ver = sys.version_info[:2]<br>
> >><br>
> >> types = (<br>
> >> bytes, bytearray, str, dict, list, tuple,<br>
> >> Enum, Counter, defaultdict, deque, OrderedDict,<br>
> >> )<br>
> >> numeric_types = int, float, Decimal, Fraction<br>
> >><br>
> >> operators = (<br>
> >> '__add__', '__and__', '__floordiv__',<br>
> >> '__iadd__', '__iand__', '__ifloordiv__', '__ilshift__',<br>
> >> '__imod__', '__imul__', '__ior__', '__ipow__',<br>
> >> '__irshift__', '__isub__', '__itruediv__', '__ixor__',<br>
> >> '__lshift__', '__mod__', '__mul__',<br>
> >> '__or__', '__pow__', '__rshift__', '__sub__', '__truediv__',<br>
> >> '__xor__',<br>
> >> )<br>
> >><br>
> >> if py_ver >= (3, 0):<br>
> >> operators += ('__gt__', '__ge__', '__le__','__lt__')<br>
> >><br>
> >> ordered_reflections = {<br>
> >> '__le__': '__ge__',<br>
> >> '__lt__': '__gt__',<br>
> >> '__ge__': '__le__',<br>
> >> '__gt__': '__lt__',<br>
> >> }<br>
> >><br>
> >><br>
> >> # helpers<br>
> >><br>
> >> class SomeOtherClass:<br>
> >> """"<br>
> >> used to test behavior when a different type is passed in to the<br>
> >> special methods<br>
> >> """<br>
> >> def __repr__(self):<br>
> >> return 'SomeOtherClass'<br>
> >> some_other_class = SomeOtherClass()<br>
> >><br>
> >> class MainClassHandled(Exception):<br>
> >> """<br>
> >> called by base class if both operands are of type base class<br>
> >> """<br>
> >><br>
> >> class SubClassCalled(Exception):<br>
> >> """<br>
> >> called by reflected operations for testing<br>
> >> """<br>
> >><br>
> >> def create_control(test_op):<br>
> >> def _any(self, other):<br>
> >> if not type(other) is self.__class__:<br>
> >> return NotImplemented<br>
> >> raise MainClassHandled<br>
> >> class Control:<br>
> >> "returns NotImplemented when other object is not supported"<br>
> >> _any.__name__ = op<br>
> >> setattr(Control, test_op, _any)<br>
> >> return Control()<br>
> >><br>
> >> def create_subtype(test_op, base_class=object):<br>
> >> def _any(*a):<br>
> >> global subclass_called<br>
> >> subclass_called = True<br>
> >> raise SubClassCalled<br>
> >> class subtype(base_class):<br>
> >> __add__ = __sub__ = __mul__ = __truediv__ = __floordiv__ = _any<br>
> >> __mod__ = __divmod__ = __pow__ = __lshift__ = __rshift__ = _any<br>
> >> __and__ = __xor__ = __or__ = _any<br>
> >> __radd__ = __rsub__ = __rmul__ = __rtruediv__ = __rfloordiv__ =<br>
> >> _any<br>
> >> __rmod__ = __rdivmod__ = __rpow__ = __rlshift__ = __rrshift__ =<br>
> >> _any<br>
> >> __rand__ = __rxor__ = __ror__ = _any<br>
> >> __le__ = __lt__ = __gt__ = __ge__ = _any<br>
> >> if issubclass(subtype, (bytes, bytearray)):<br>
> >> value = b'hello'<br>
> >> elif issubclass(subtype, str):<br>
> >> value = 'goodbye'<br>
> >> elif issubclass(subtype, (list, tuple)):<br>
> >> value = (1, 2, 3)<br>
> >> elif issubclass(subtype, (int, float, Decimal, Fraction)):<br>
> >> value = 42<br>
> >> else:<br>
> >> # ignore value<br>
> >> return subtype()<br>
> >> return subtype(value)<br>
> >><br>
> >><br>
> >> # test exceptions<br>
> >><br>
> >> # control against some other class<br>
> >> print('testing control...\n')<br>
> >> errors = False<br>
> >> for op in operators:<br>
> >> control = create_control(op)<br>
> >> op = getattr(operator, op)<br>
> >> try:<br>
> >> op(control, some_other_class)<br>
> >> except TypeError:<br>
> >> # the end result of no method existing, or each method called<br>
> >> returning<br>
> >> # NotImplemented because it does not know how to perform the<br>
> >> requested<br>
> >> # operation between the two types<br>
> >> pass<br>
> >> except Exception as exc:<br>
> >> errors = True<br>
> >> print('%s(%s()) -- Exception <%s> raised instead of TypeError' %<br>
> >> (op.__name__, test_type.__name__, exc))<br>
> >> else:<br>
> >> errors = True<br>
> >> print('Control -- TypeError not raised for op %r' % op)<br>
> >> if errors:<br>
> >> print('errors in Control -- misunderstanding or bug?\n')<br>
> >><br>
> >> # control against a subclass<br>
> >> errors = False<br>
> >> for op in operators:<br>
> >> subclass_called = False<br>
> >> control = create_control(op)<br>
> >> subtype = create_subtype(op, control.__class__)<br>
> >> op = getattr(operator, op)<br>
> >> try:<br>
> >> op(control, subtype)<br>
> >> except SubClassCalled:<br>
> >> # if the control class properly signals that it doesn't know how<br>
> >> to<br>
> >> # perform the operation, of if Python notices that a reflected<br>
> >> # operation exists, we get here (which is good)<br>
> >> pass<br>
> >> except MainClassHandled:<br>
> >> errors = True<br>
> >> print('Control did not yield to subclass for op %r' % op)<br>
> >> except Exception as exc:<br>
> >> if subclass_called:<br>
> >> # exception was subverted to something more appropriate (like<br>
> >> # unorderable types)<br>
> >> pass<br>
> >> errors = True<br>
> >> print('%s -- Exception <%s> raised' %<br>
> >> (op.__name__, exc))<br>
> >> else:<br>
> >> errors = True<br>
> >> print('Control -- op %r appears to have succeeded (it should not<br>
> >> have)' % op)<br>
> >> if errors:<br>
> >> print('errors in Control -- misunderstanding or bug?\n')<br>
> >><br>
> >><br>
> >> # tests<br>
> >> print('testing types against a foreign class\n')<br>
> >> for test_type in types + numeric_types:<br>
> >> errors = False<br>
> >> for op in operators:<br>
> >> op = getattr(operator, op)<br>
> >> try:<br>
> >> op(test_type(), some_other_class)<br>
> >> except TypeError:<br>
> >> pass<br>
> >> except Exception as exc:<br>
> >> errors = True<br>
> >> print('%s(%s()) -- Exception <%s> raised instead of<br>
> >> TypeError' %<br>
> >> (op.__name__, test_type.__name__, exc))<br>
> >> else:<br>
> >> print('%s(%s()) -- NotImplemented not returned, TypeError<br>
> >> not raised' %<br>
> >> (op.__name__, test_type.__name__))<br>
> >> if errors:<br>
> >> print()<br>
> >><br>
> >> print()<br>
> >><br>
> >> # test subclasses<br>
> >> print('testing types against a subclass\n')<br>
> >> for test_type in types:<br>
> >> errors = False<br>
> >> for op in operators:<br>
> >> subclass_called = False<br>
> >> if not test_type.__dict__.get(op):<br>
> >> continue<br>
> >> subclass = create_subtype(op, test_type)<br>
> >> op = getattr(operator, op)<br>
> >> try:<br>
> >> if test_type is str:<br>
> >> op('%s', subtype)<br>
> >> else:<br>
> >> op(test_type(), subtype)<br>
> >> except SubClassCalled:<br>
> >> # expected, ignore<br>
> >> pass<br>
> >> except Exception as exc:<br>
> >> if subclass_called:<br>
> >> # exception raised by subclass was changed<br>
> >> pass<br>
> >> errors = True<br>
> >> print('%s(%s()) -- Exception <%s> raised (should have<br>
> >> worked)' %<br>
> >> (op.__name__, test_type.__name__, exc))<br>
> >> else:<br>
> >> errors = True<br>
> >> print('%s(%s()) -- NotImplemented not returned, TypeError<br>
> >> not raised' %<br>
> >> (op.__name__, test_type.__name__))<br>
> >> if errors:<br>
> >> print()<br>
> >> for test_type in numeric_types:<br>
> >> errors = False<br>
> >> for op in operators:<br>
> >> subclass_called = False<br>
> >> if not test_type.__dict__.get(op):<br>
> >> continue<br>
> >> subtype = create_subtype(op, test_type)<br>
> >> op = getattr(operator, op)<br>
> >> try:<br>
> >> op(test_type(), subtype)<br>
> >> except SubClassCalled:<br>
> >> # expected, ignore<br>
> >> pass<br>
> >> except Exception as exc:<br>
> >> if subclass_called:<br>
> >> # exception raised by subclass was changed<br>
> >> pass<br>
> >> errors = True<br>
> >> print('%s(%s()) -- Exception <%s> raised (should have<br>
> >> worked)' %<br>
> >> (op.__name__, test_type.__name__, exc))<br>
> >> else:<br>
> >> errors = True<br>
> >> print('%s(%s)) -- NotImplemented not returned' %<br>
> >> (op.__name__, test_type.__name__))<br>
> >> if errors:<br>
> >> print()<br>
> >> -- 8< ------------------------------------------------------------<br>
> >> ----------------<br>
> >> _______________________________________________<br>
> >> Python-Dev mailing list<br>
> >> <a href="mailto:Python-Dev@python.org">Python-Dev@python.org</a><br>
> >> <a href="https://mail.python.org/mailman/listinfo/python-dev" target="_blank">https://mail.python.org/mailman/listinfo/python-dev</a><br>
> >> Unsubscribe: <a href="https://mail.python.org/mailman/options/python-dev/" target="_blank">https://mail.python.org/mailman/options/python-dev/</a><br>
> >> brett%<a href="http://40python.org" target="_blank">40python.org</a><br>
> >><br>
> ><br>
> > _______________________________________________<br>
> > Python-Dev mailing list<br>
> > <a href="mailto:Python-Dev@python.org">Python-Dev@python.org</a><br>
> > <a href="https://mail.python.org/mailman/listinfo/python-dev" target="_blank">https://mail.python.org/mailman/listinfo/python-dev</a><br>
> > Unsubscribe:<br>
> > <a href="https://mail.python.org/mailman/options/python-dev/guido%40python.org" target="_blank">https://mail.python.org/mailman/options/python-dev/guido%40python.org</a><br>
> ><br>
> ><br>
><br>
<br>
<br>
_______________________________________________<br>
Python-Dev mailing list<br>
<a href="mailto:Python-Dev@python.org">Python-Dev@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/python-dev" target="_blank">https://mail.python.org/mailman/listinfo/python-dev</a><br>
Unsubscribe: <a href="https://mail.python.org/mailman/options/python-dev/guido%40python.org" target="_blank">https://mail.python.org/mailman/options/python-dev/guido%40python.org</a><br>
</blockquote></div>