[issue4155] Wrong math calculation

[Raymond Hettinger]

Raymond Hettinger <rhettinger at users.sourceforge.net> added the comment:

None of these calculations are going to work out exactly.  Look-up "what
every computer scientist needs to know about floating point" and see the
floating-point appendix in the Python tutorial.  In this case, the
difference is only one bit in the last place:

  >>> 0.70710678118654746.hex()
  '0x1.6a09e667f3bccp-1'
  >>> 0.70710678118654757.hex()
  '0x1.6a09e667f3bcdp-1'

----------
nosy: +rhettinger
resolution:  -> invalid
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue4155>
_______________________________________