I'm trying to understand some unexpected behaviour with float NANs in Python 3.5. Background: in IEEE-754 maths, NANs (Not A Number) come in two flavours, so called "quiet NANs" and "signalling NANs". By default, arithmetic operations on qnans return a qnan; operations on snans "signal", which in Python terms means raising an exception. The original IEEE-754 standard didn't specify how to distinguish a qnan from a snan, but a de facto standard arose that bit 51 of the float was the "quiet bit", if it were set, the NAN was quiet. For the purposes of this email, I'm going to assume that standard is in place, even though technically speaking it is platform-dependent. According to my tests, it seems that we cannot create snans in Python. The float constructor doesn't recognise "snan", raising ValueError. Nor can we convert a Decimal snan into a float: py> from decimal import Decimal py> snan = Decimal('snan') py> float(dsnan) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: cannot convert signaling NaN to float But unexpectedly (to me at least), we apparently cannot even create a signalling NAN by casting 64 bits to a float. Here are the functions I use to do the cast: from struct import pack, unpack def cast_float2int(x): return unpack('<Q', pack('<d', x))[0] def cast_int2float(i): return unpack('<d', pack('<Q', i))[0] Here's a regular quiet NAN round-tripping, as expected: py> x = cast_int2float(0x7ff8000000000001) py> x nan py> hex(cast_float2int(x)) '0x7ff8000000000001' So far so good. But now let me try with a signalling NAN: py> x = cast_int2float(0x7ff0000000000001) py> x nan py> hex(cast_float2int(x)) '0x7ff8000000000001' So it seems that the "quiet" bit is automatically set, even when using the struct module, making it impossible to create snan floats. Is this intended? If so, why? Is this meant as a language feature? Thanks in advance, Steve
works for me, too: In [9]: x = cast_int2float(0x7ff8000000000001) In [10]: hex(cast_float2int(x)) Out[10]: '0x7ff8000000000001' In [11]: x = cast_int2float(0x7ff0000000000001) In [12]: hex(cast_float2int(x)) Out[12]: '0x7ff0000000000001' OS-X, conda build: Python 3.7.0 | packaged by conda-forge | (default, Aug 27 2018, 17:24:52) [Clang 6.1.0 (clang-602.0.53)] on darwin I suspect it depends on the compiler's math library But neither is raising an exception: In [3]: x = cast_int2float(0x7ff0000000000001) In [4]: y = cast_int2float(0x7ff8000000000001) In [5]: 2.0 / x Out[5]: nan In [6]: 2.0 / y Out[6]: nan When should it? -CHB On Fri, Nov 9, 2018 at 3:46 AM, Serhiy Storchaka <storchaka@gmail.com> wrote:
09.11.18 13:05, Steven D'Aprano пише:
py> x = cast_int2float(0x7ff0000000000001) py> x nan py> hex(cast_float2int(x)) '0x7ff8000000000001'
I got '0x7ff0000000000001'.
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/chris. barker%40noaa.gov
-- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Fri, Nov 09, 2018 at 01:17:09PM -0800, Chris Barker via Python-Dev wrote:
works for me, too:
In [9]: x = cast_int2float(0x7ff8000000000001) In [10]: hex(cast_float2int(x)) Out[10]: '0x7ff8000000000001'
In [11]: x = cast_int2float(0x7ff0000000000001) In [12]: hex(cast_float2int(x)) Out[12]: '0x7ff0000000000001'
Fascinating. I borrowed a Debian system and tried it on there, and got the same results as you. So I wonder whether it is something unusual about my Red Hat system that it prevents the formation of signalling NANs? However, I don't think that explains why the float constructor doesn't allow Decimal('snan') to be converted to a float.
I suspect it depends on the compiler's math library
Unfortunately that's probably true.
But neither is raising an exception: [...] When should it?
I think that, by default, any arithmetic operation, comparison or math library function call ought to raise if given a snan, if the underlying math library supports IEEE-754 signals. Which I imagine these days nearly all should do. So any of these should raise: snan + 1 math.sin(snan) snan == 0 -- Steve
On Sat, Nov 10, 2018 at 3:26 PM, Steven D'Aprano <steve@pearwood.info> wrote:
On Fri, Nov 09, 2018 at 01:17:09PM -0800, Chris Barker via Python-Dev wrote:
works for me, too:
In [9]: x = cast_int2float(0x7ff8000000000001) In [10]: hex(cast_float2int(x)) Out[10]: '0x7ff8000000000001'
In [11]: x = cast_int2float(0x7ff0000000000001) In [12]: hex(cast_float2int(x)) Out[12]: '0x7ff0000000000001'
Fascinating. I borrowed a Debian system and tried it on there, and got the same results as you. So I wonder whether it is something unusual about my Red Hat system that it prevents the formation of signalling NANs?
Apparently loading a sNaN into an x87 register silently converts it to a qNaN, and on Linux C compilers are allowed to do that at any point: https://stackoverflow.com/questions/22816095/signalling-nan-was-corrupted-wh... So the Debian/RH difference may just be different register allocation in two slightly different compiler versions. Also, gcc doesn't even try to support sNaN correctly unless you pass -fsignaling-nans, and the docs warn that this isn't very well tested: https://www.cleancss.com/explain-command/gcc/5197 IEEE754 is a wonderful thing, and even imperfect implementations are still way better than what came before, but real systems are almost universally sloppy about details. I don't think any real libm even tries to set all the status flags correctly. And I'm not sure sNaN support is actually useful anyway... Of course if all you want is a value that raises an exception whenever it's used in an arithmetic expression, then Python does support that :-) sNaN = object() -n -- Nathaniel J. Smith -- https://vorpus.org
On Sat, Nov 10, 2018 at 04:27:29PM -0800, Nathaniel Smith wrote:
Apparently loading a sNaN into an x87 register silently converts it to a qNaN, and on Linux C compilers are allowed to do that at any point:
https://stackoverflow.com/questions/22816095/signalling-nan-was-corrupted-wh...
Thanks for finding that.
So the Debian/RH difference may just be different register allocation in two slightly different compiler versions.
The Debian box uses an ARM processor, so there's that difference too. -- Steve
participants (5)
-
Chris Barker -
Greg Ewing -
Nathaniel Smith -
Serhiy Storchaka -
Steven D'Aprano