[issue14245] float rounding examples in FAQ are outdated

[Mark Dickinson]

Mark Dickinson <dickinsm at gmail.com> added the comment:

> I think it would be useful to mention explicitly that Python simply uses 
> the native floating-point implementation in hardware and thus behaves 
> very similarly to other languages which do this, for instance C or Java. 

Agreed that it's useful to say something here to make it obvious that Python's behaving just like most other mainstream programming languages.  Perhaps most straightforward would be to state that CPython just uses C doubles.  (That sidesteps nitpicking about software floating-point :-).

> It is easy to count, that exactly 17 digits are accurate.

Hmm;  that depends a bit on your definitions.  The relative error here is around 3.7e-17, so in this particular case you're getting something between 16 and 17 decimal digits of accuracy.  Other cases would produce different relative errors, with a max relative error for normal cases of 2**-53, or approximately 1.1e-16.

> I have to admit, that I'm completely lost here --- why would a vastly 
> inaccurate number (with more than half of digits wrong) be ever stored?

Because that number is the unique closest representable IEEE 754 double to the target number (1.2).  You can go through all the steps here:

 - convert 1.2 to binary, to get:

    1.001100110011001100110011.....  repeated ad infinitum

 - round to the nearest 53-bit number:

    1.0011001100110011001100110011001100110011001100110011

 - convert back to decimal to get the number that I gave.

But this sort of detail is already there in the tutorial, and doesn't seem appropriate for the FAQ entry.  For the FAQ, maybe it would be less confusing to give only about a 20-digit approximation.

> "Similarly, the result of any
> floating-point operation must often be rounded to fit into the 
> internal format,
> resulting in another tiny error." ?

The aim here was to make the point that it's not just conversion from decimal to binary that introduces errors;  it's also true that any arithmetic operation has the potential to introduce errors.  For example, 10**16 and 1 are both exactly representable as floats, but the sum of 10**16 + 1 is not;  the result has to be rounded to fit into a float.  Suggestions for rewording to make this clearer are welcome!

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue14245>
_______________________________________