[docs] [issue14245] float rounding examples in FAQ are outdated
report at bugs.python.org
Sat Mar 10 15:17:02 CET 2012
Zbyszek Szmek <zbyszek at in.waw.pl> added the comment:
[part mangled by the tracker]
"> which is accurate to around 16 decimal digits.)
It is easy to count, that exactly 17 digits are accurate.
I have to admit, that I'm completely lost here --- why would a vastly
inaccurate number (with more than half of digits wrong) be ever stored?
If "1.2" is converted to a float (a C double in current implementation),
it has 15.96 decimal digits of precision.
" > Similarly, the result of any
" > floating-point operation must often be rounded to fit into the
" > internal format, resulting in another tiny error.
"Similarly, the result of a floating-point operation must be rounded to
fit into the fixed precision, often resulting in another tiny error." ?
Python tracker <report at bugs.python.org>
More information about the docs