int() and math.trunc don't accept objects that only define __index__

Hi, I open this thread to discuss the proposal by Nick Coghlan in https://bugs.python.org/issue33039 to add __int__ and __trunc__ to a type when __index__ is defined. Currently __int__ does not default to __index__ during class initialisation so both must be defined to get a coherant behavior: (cpython-venv) ➜ cpython git:(add-key-argument-to-bisect) ✗ python3 Python 3.8.0a1+ (heads/add-key-argument-to-bisect:b7aaa1adad, Feb 18 2019, 16:10:22) [Clang 10.0.0 (clang-1000.10.44.4)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import math >>> class MyInt: ... def __index__(self): ... return 4 ... >>> int(MyInt()) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: int() argument must be a string, a bytes-like object or a number, not 'MyInt' >>> math.trunc(MyInt()) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: type MyInt doesn't define __trunc__ method >>> hex(MyInt()) '0x4' >>> len("a"*MyInt()) 4 >>> MyInt.__int__ = MyInt.__index__ >>> int(MyInt()) 4 The difference in behavior is espacially weird in builtins like int() and hex(). The documentation mentions at https://docs.python.org/3/reference/datamodel.html#object.__index__ the need to always define both __index__ and __int__: Note: In order to have a coherent integer type class, when __index__() is defined __int__() should also be defined, and both should return the same value. Nick Coghlan proposes to make __int__ defaults to __index__ when only the second is defined and asked to open a discussion on python-dev before making any change "as the closest equivalent we have to this right now is the "negative" derivation, where overriding __eq__ without overriding __hash__ implicitly marks the derived class as unhashable (look for "type->tp_hash = PyObject_HashNotImplemented;").". I think the change proposed makes more sense than the current behavior and volunteer to implement it if it is accepted. What do you think about this?

On Tue, 19 Feb 2019 at 03:31, Rémi Lapeyre <remi.lapeyre@henki.fr> wrote:
Reading this again now, it occurs to me that there's another developer experience improvement we already made along these lines in Python 3: "By default, __ne__() delegates to __eq__() and inverts the result unless it is NotImplemented. " [1] By contrast, the corresponding (and annoying) Python 2 behaviour was: "The truth of x==y does not imply that x!=y is false. Accordingly, when defining __eq__(), one should also define __ne__() so that the operators will behave as expected." [2] The only difference is that whereas the new `__ne__` delegation behaviour could just be defined directly in `object.__ne__()`, `object` doesn't implement `__int__` by default, so the delegating function would need to be injected into the type when it is defined (and that's the part that's similar to the `__hash__ = None` negative derivation). So +1 from me. Cheers, Nick. [1] https://docs.python.org/3/reference/datamodel.html#object.__ne__ [2] https://docs.python.org/2/reference/datamodel.html#object.__ne__ -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Another point in favor of the change I just noticed is that int() accept objects defining __index__ as its `base` argument: Python 3.7.2 (default, Jan 13 2019, 12:50:01) [Clang 10.0.0 (clang-1000.11.45.5)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> class MyInt: ... def __index__(self): ... return 4 ... >>> int("3", base=MyInt()) 3 >>> int(MyInt()) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: int() argument must be a string, a bytes-like object or a number, not 'MyInt'

18.02.19 18:16, Rémi Lapeyre пише:
Should we add default implementations of __float__ and __complex__ when either __index__ or __int__ is defined? Currently:
Or just document that in order to have a coherent integer type class, when __index__() or __int__() are defined __float__() and __complex__() should also be defined, and all should return equal values.

On Fri, 22 Feb 2019 at 18:29, Serhiy Storchaka <storchaka@gmail.com> wrote:
I think when __index__ is defined, it would be reasonable to have that imply the same floating point conversion rules as are applied for builtin ints, since the conversion is supposed to be lossless in that case (and if it isn't lossless, that's what `__int__` is for). However, I don't think the decision is quite as clearcut as it is for `__index__` implying `__int__`. Lossy conversions to int shouldn't imply anything about conversions to real numbers or floating point values. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Le 22 février 2019 à 18:14:01, Nick Coghlan (ncoghlan@gmail.com(mailto:ncoghlan@gmail.com)) a écrit:
When __index__ is defined it means that there is a lossless conversion to int possible. In this case, this means a lossless conversion to float and complex is also possible (with the exception of overflows but anyone doing float(var) should expect them). In this case, I think defining __index__ could set a default for __int__, __float__ and __complex__.
Lossy conversions to int shouldn't imply anything about conversions to real numbers or floating point values.
I think the right behavior is the other way around, it should go the other way around from a superset to a subset, when __int__ is defined __float__ should not have a default set because there might be a better default than float(int(self)), but when __float__ is defined, it should be safe to make __int__ default to int(float(self)) if not defined. This would simplify this behavior: Python 3.7.2 (default, Jan 13 2019, 12:50:01) [Clang 10.0.0 (clang-1000.11.45.5)] on darwin Type "help", "copyright", "credits" or "license" for more information.
Defining __complex__ shouldn’t do anything since they have one more dimension there is no clear way to convert to ints and floats which is already the behavior of the the builtin complex type:
Here’s an implementation of what I’m thinking of:

On Wed, Mar 13, 2019 at 03:21:31AM -0700, Rémi Lapeyre wrote:
That's not correct: py> n = 2**64 + 1 py> n == int(float(n)) False Python floats (C doubles) can lose digits when converting from ints over 2**53 or so.
(with the exception of overflows but anyone doing float(var) should expect them).
I don't. I expect float(var) to overflow to infinity, if it is going to overflow, and always forget that it can raise. py> float("9e9999") inf py> float(str(9*10**9999)) inf But: py> float(9*10**9999) Traceback (most recent call last): File "<stdin>", line 1, in <module> OverflowError: int too large to convert to float This never fails to surprise me. -- Steven

On Tue, 19 Feb 2019 at 03:31, Rémi Lapeyre <remi.lapeyre@henki.fr> wrote:
Reading this again now, it occurs to me that there's another developer experience improvement we already made along these lines in Python 3: "By default, __ne__() delegates to __eq__() and inverts the result unless it is NotImplemented. " [1] By contrast, the corresponding (and annoying) Python 2 behaviour was: "The truth of x==y does not imply that x!=y is false. Accordingly, when defining __eq__(), one should also define __ne__() so that the operators will behave as expected." [2] The only difference is that whereas the new `__ne__` delegation behaviour could just be defined directly in `object.__ne__()`, `object` doesn't implement `__int__` by default, so the delegating function would need to be injected into the type when it is defined (and that's the part that's similar to the `__hash__ = None` negative derivation). So +1 from me. Cheers, Nick. [1] https://docs.python.org/3/reference/datamodel.html#object.__ne__ [2] https://docs.python.org/2/reference/datamodel.html#object.__ne__ -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Another point in favor of the change I just noticed is that int() accept objects defining __index__ as its `base` argument: Python 3.7.2 (default, Jan 13 2019, 12:50:01) [Clang 10.0.0 (clang-1000.11.45.5)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> class MyInt: ... def __index__(self): ... return 4 ... >>> int("3", base=MyInt()) 3 >>> int(MyInt()) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: int() argument must be a string, a bytes-like object or a number, not 'MyInt'

18.02.19 18:16, Rémi Lapeyre пише:
Should we add default implementations of __float__ and __complex__ when either __index__ or __int__ is defined? Currently:
Or just document that in order to have a coherent integer type class, when __index__() or __int__() are defined __float__() and __complex__() should also be defined, and all should return equal values.

On Fri, 22 Feb 2019 at 18:29, Serhiy Storchaka <storchaka@gmail.com> wrote:
I think when __index__ is defined, it would be reasonable to have that imply the same floating point conversion rules as are applied for builtin ints, since the conversion is supposed to be lossless in that case (and if it isn't lossless, that's what `__int__` is for). However, I don't think the decision is quite as clearcut as it is for `__index__` implying `__int__`. Lossy conversions to int shouldn't imply anything about conversions to real numbers or floating point values. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Le 22 février 2019 à 18:14:01, Nick Coghlan (ncoghlan@gmail.com(mailto:ncoghlan@gmail.com)) a écrit:
When __index__ is defined it means that there is a lossless conversion to int possible. In this case, this means a lossless conversion to float and complex is also possible (with the exception of overflows but anyone doing float(var) should expect them). In this case, I think defining __index__ could set a default for __int__, __float__ and __complex__.
Lossy conversions to int shouldn't imply anything about conversions to real numbers or floating point values.
I think the right behavior is the other way around, it should go the other way around from a superset to a subset, when __int__ is defined __float__ should not have a default set because there might be a better default than float(int(self)), but when __float__ is defined, it should be safe to make __int__ default to int(float(self)) if not defined. This would simplify this behavior: Python 3.7.2 (default, Jan 13 2019, 12:50:01) [Clang 10.0.0 (clang-1000.11.45.5)] on darwin Type "help", "copyright", "credits" or "license" for more information.
Defining __complex__ shouldn’t do anything since they have one more dimension there is no clear way to convert to ints and floats which is already the behavior of the the builtin complex type:
Here’s an implementation of what I’m thinking of:

On Wed, Mar 13, 2019 at 03:21:31AM -0700, Rémi Lapeyre wrote:
That's not correct: py> n = 2**64 + 1 py> n == int(float(n)) False Python floats (C doubles) can lose digits when converting from ints over 2**53 or so.
(with the exception of overflows but anyone doing float(var) should expect them).
I don't. I expect float(var) to overflow to infinity, if it is going to overflow, and always forget that it can raise. py> float("9e9999") inf py> float(str(9*10**9999)) inf But: py> float(9*10**9999) Traceback (most recent call last): File "<stdin>", line 1, in <module> OverflowError: int too large to convert to float This never fails to surprise me. -- Steven
participants (4)
-
Nick Coghlan
-
Rémi Lapeyre
-
Serhiy Storchaka
-
Steven D'Aprano