round issue

Robin Becker robin at reportlab.com
Mon Jul 12 05:52:00 EDT 2010


A client wants to know why his db number -9.85 gets displayed by some simple 
code as -9.8

I looked at the number and see that

 >>> -9.85
-9.8499999999999996

ie I expect simple rounding to produce the observed result and indeed

 >>> '%.1f' % -9.85
'-9.8'

however, when I use round I get an unexpected result ie
 >>> round(-9.85,1)
-9.9000000000000004

according to its definition

> round(x[, n])¶
>     Return the floating point value x rounded to n digits after the decimal point.
>     If n is omitted, it defaults to zero. The result is a floating point number.
>     Values are rounded to the closest multiple of 10 to the power minus n;
>     if two multiples are equally close, rounding is done away from 0 (so. for example,
>     round(0.5) is 1.0 and round(-0.5) is -1.0).

so looking at the absolute differences I see

 >>> abs(-9.9 - -9.85)
0.050000000000000711
 >>> abs(-9.8 - -9.85)
0.049999999999998934

ie the -9.8 value appears closer and at least to a primitive test

 >>> abs(-9.9 - -9.85) > abs(-9.8 - -9.85)
True

the distance from the -9.9 result is larger, however, that may be because the 
model numbers for -9.8 & -9.9 differ in distance from the true 10**-n values eg

 >>> -9.9
-9.9000000000000004
 >>> -9.8
-9.8000000000000007

What value should round(-9.85,1) return? Is the result explainable in python (ie 
without resort to the internal FP representations etc etc)?
-- 
Robin Becker




More information about the Python-list mailing list