[Python-Dev] Rounding float to int directly ...

Michael Chermside mcherm at mcherm.com
Wed Aug 2 13:18:34 CEST 2006


Nick Maclaren:
> The "wobbling precision" effect may be overstated,
> but is real, and gets worse the larger the base is.  To the best of my
> knowledge, that is almost the only way in which binary is more accurate
> than decimal, in absolute terms, and it is a marginal difference.

Okay, that makes sense.

However, if I understand "wobbling precision" correctly (and it's a BIG
if) then it refers to the fact that interpolating from the last digit of
precision doesn't work so well (intentionally vague phrase). So I
would think (but could be wrong) that it is always trumped by adding
another digit of precision to your numbers / lookup tables / whatever.
If so, then even more credit needs to be given to the system that
supports adjustable precision (the Decimal module).

-- Michael Chermside



More information about the Python-Dev mailing list