round issue

Gary Herron gherron at islandtraining.com
Mon Jul 12 12:42:25 EDT 2010


On 07/12/2010 02:52 AM, Robin Becker wrote:
> A client wants to know why his db number -9.85 gets displayed by some 
> simple code as -9.8
>
> I looked at the number and see that
>
> >>> -9.85
> -9.8499999999999996
>
> ie I expect simple rounding to produce the observed result and indeed
>
> >>> '%.1f' % -9.85
> '-9.8'
>
> however, when I use round I get an unexpected result ie
> >>> round(-9.85,1)
> -9.9000000000000004
>
> according to its definition
>
>> round(x[, n])¶
>>     Return the floating point value x rounded to n digits after the 
>> decimal point.
>>     If n is omitted, it defaults to zero. The result is a floating 
>> point number.
>>     Values are rounded to the closest multiple of 10 to the power 
>> minus n;
>>     if two multiples are equally close, rounding is done away from 0 
>> (so. for example,
>>     round(0.5) is 1.0 and round(-0.5) is -1.0).
>
> so looking at the absolute differences I see
>
> >>> abs(-9.9 - -9.85)
> 0.050000000000000711
> >>> abs(-9.8 - -9.85)
> 0.049999999999998934

Do you *seriously* need to ask a computer if -9.85 is half-way between 
-9.9 and -9.8.   Just look at the numbers man!   Of course it's equally 
close to both, so rounding to -9.9 is correct according to the definition.



Do you actually *believe* that -9.9  -  -9.85 = 0.050000000000000711.    
Of course you know it's really 0.05.  All you've done here is 
demonstrate the fallible nature of (the last several digits of) floating 
point arithmetic on a computer.   Don't let that distract you from using 
your common sense.


Gary Herron




> ie the -9.8 value appears closer and at least to a primitive test
>
> >>> abs(-9.9 - -9.85) > abs(-9.8 - -9.85)
> True
>
> the distance from the -9.9 result is larger, however, that may be 
> because the model numbers for -9.8 & -9.9 differ in distance from the 
> true 10**-n values eg
>
> >>> -9.9
> -9.9000000000000004
> >>> -9.8
> -9.8000000000000007
>
> What value should round(-9.85,1) return? Is the result explainable in 
> python (ie without resort to the internal FP representations etc etc)?





More information about the Python-list mailing list