Inconsistent results from int(floatNumber)
Terry Reedy
tjreedy at udel.edu
Tue Oct 26 00:25:23 CEST 2010
On 10/25/2010 5:44 PM, gershar wrote:
> I had some problems with some Python projects that gave variable
> results that I could not track down. Eventually and reluctantly I
> converted them to Java. Later, when I had more time I tried to analyze
> what the Python code was doing and found something strange. The
> following snippet illustrates the problem.
>
>>>> i = -50.0
>>>> for x in xrange(5):
> i += 0.1
The binary float resulting from the conversion of .1 is slightly greater
than .1, so this increases i by slightly more than .1
> z = i * 10.0
so z is increased be lightly more that 1
> print
> print z
> print int(z)
float.__int__ truncates toward 0.
> -499.0
> -499
>
> -498.0
> -498
>
> -497.0
> -496
And here the extra increase shows up.
> -496.0
> -495
>
> -495.0
> -494
> It looks like a rounding problem but on the surface there is nothing
> to round. I am aware that there are rounding limitations with floating
> point arithmetic but the value passed to int() is always correct.
No it is not. To see this, print more digits (which themselves are
approximations of the actual binary value). With 3.1.2 on x86 system:
i = -50.0
form = '{:24.18f}'.format
print(form(.1))
for x in range(15):
i += 0.1
z = i * 10.0
print()
print(form(z))
print(int(z))
>>>
0.100000000000000006
-499.000000000000000000
-499
-498.000000000000000000
-498
-496.999999999999943157
-496
-495.999999999999943157
-495
-494.999999999999943157
-494
To completely understand, you would have to look at the binary bit
pattern and know the exact behavior of floating point arithmetic on a
system and the exact decimal to binary to decimal conversion algorithm.
--
Terry Jan Reedy
More information about the Python-list
mailing list