Re: [Python-Dev] Rounding float to int directly ...
Nick Maclaren:
The "wobbling precision" effect may be overstated, but is real, and gets worse the larger the base is. To the best of my knowledge, that is almost the only way in which binary is more accurate than decimal, in absolute terms, and it is a marginal difference.
Okay, that makes sense. However, if I understand "wobbling precision" correctly (and it's a BIG if) then it refers to the fact that interpolating from the last digit of precision doesn't work so well (intentionally vague phrase). So I would think (but could be wrong) that it is always trumped by adding another digit of precision to your numbers / lookup tables / whatever. If so, then even more credit needs to be given to the system that supports adjustable precision (the Decimal module). -- Michael Chermside
participants (1)
-
Michael Chermside