[issue39111] Misleading documentation
New submission from Murali Ganapathy <murali@google.com>: The documentation at https://docs.python.org/3.6/library/constants.html#NotImplemented states If all attempts return NotImplemented, the interpreter will raise an appropriate exception. However this is not true for __eq__. === class Foo: def __eq__(self, other): return NotImplemented Foo() == Foo() # returns False, does not throw an exception ==== ---------- assignee: docs@python components: Documentation messages: 358719 nosy: docs@python, murali priority: normal severity: normal status: open title: Misleading documentation versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue39111> _______________________________________
Brett Cannon <brett@python.org> added the comment: This is because your class implicitly inherits from object and object.__eq__() is implemented and does not return NotImplemented. ---------- nosy: +brett.cannon resolution: -> not a bug stage: -> resolved status: open -> closed _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue39111> _______________________________________
Brett Cannon <brett@python.org> added the comment: And to be more specific, == is guaranteed to work by falling back to object.__eq__() which falls back to object identity if the object doesn't have a custom __eq__() method. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue39111> _______________________________________
Murali Ganapathy <murali@google.com> added the comment: === # python3 class Base: def __eq__(self, other): print("base called") return super().__eq__(other) class Foo(Base): def __eq__(self, other): print("foo called") return NotImplemented Foo() == Foo() # foo called # foo called False ==== Base.__eq__ is not called here. Is calling of object.__eq__ special cased? ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue39111> _______________________________________
Steven D'Aprano <steve+python@pearwood.info> added the comment: The behaviour is correct: `==` should not raise, even if both objects' `__eq__` method returns NotImplemented. I agree that the documentation needs improvement. Looking at the current version: https://docs.python.org/3/library/constants.html#NotImplemented it says: Note: When a binary (or in-place) method returns NotImplemented the interpreter will try the reflected operation on the other type (or some other fallback, depending on the operator). If all attempts return NotImplemented, the interpreter will raise an appropriate exception. Incorrectly returning NotImplemented will result in a misleading error message or the NotImplemented value being returned to Python code. (1) In the case of `==` and `!=`, an exception is not raised even all relevant methods return NotImplemented. The source code for PyObject_RichCompare in object.c says: /* If neither object implements it, provide a sensible default for == and !=, but raise an exception for ordering. */ and then falls back to object identity. If we wanted to lawyer up, the current docs don't say that "some other fallback" is limited to Python dunder methods. But that's the strong impression it gives, and it is actively misleading in that there's no hint that the fallback includes a default notion of equality as object identity, built right into the interpreter itself. Even Brett got this wrong: object.__eq__ can return NotImplemented, so it isn't a suitable final fallback: py> object.__eq__(o, None) NotImplemented so I think these docs could do with some improvement. Based on the current docs, I too would expect equality to raise an exception for a class that defines `__eq__` and `__ne__` to always return NotImplemented. (2) The last sentence, about "incorrectly returning NotImplemented", confuses me. Why are we warning about that? If you incorrectly return any value, whether it is NotImplemented, True, False, None or 3.1415, it will result in a misleading error message or incorrect value being returned. Is it perhaps meant to say "incorrectly *raising* NotImplemented"? I'm reopening this for a doc enhancement. Unfortunately I can't use github for technical reasons, so can't do a PR, but I'll suggest an updated description for the first part. (I have no idea what to do for the second part.) When a binary (or in-place) method returns NotImplemented the interpreter will try the reflected operation on the other type (or some other fallback, depending on the operator). If all attempts return NotImplemented, the interpreter will fall back to object identity for `==` and `!=` or raise an appropriate exception for all other comparisons. ---------- nosy: +steven.daprano resolution: not a bug -> stage: resolved -> status: closed -> open type: -> enhancement _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue39111> _______________________________________
Change by Steven D'Aprano <steve+python@pearwood.info>: ---------- title: Misleading documentation -> Misleading documentation for NotImplemented _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue39111> _______________________________________
Change by Brett Cannon <brett@python.org>: ---------- nosy: -brett.cannon _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue39111> _______________________________________
Change by Terry J. Reedy <tjreedy@udel.edu>: ---------- versions: -Python 3.5, Python 3.6 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue39111> _______________________________________
participants (4)
-
Brett Cannon
-
Murali Ganapathy
-
Steven D'Aprano
-
Terry J. Reedy