HELP: restore my faith in Python

Johann Hibschman johann at physics.berkeley.edu
Fri Mar 3 20:30:16 CET 2000

```Holton, Steven [NCRTP:6125:EXCH] writes:
> Can anyone explaing to me why index 6 and 7 aren't what I'm expecting
> them to be?

> 6 / 5.0 is 1.2   (when we're dealing with floats)
> 6 / 5 is 1       (when we're dealing with ints)
> 1.2 - 1 = 0.2
> 0.2 * 5 = 1.0
> int( 1.0 ) = 1   (don't it?)

You're hitting rounding error.  Floats are inherently imprecise; if
you want to pull tricks like this, you should either use a rational
class, or include an allowed "epsilon" for rounding to 1.

You can see what's going on by asking python to show you more decimal
places:

>>> "%.20f" % (6/5.0)
'1.19999999999999995559'

>>> "%.20f" % ((6/5.0)-1.0)
'0.19999999999999995559'

>>> "%.20f" % (((6/5.0)-1.0)*5)
'0.99999999999999977796'

0.999... is less than 1, so int truncates it back down to 0.

In short, you can't trust floats to give precise answers, if the last
few decimal places matter.  If you're truncating to int, rather than
rounding, then those last few places do matter.  This is true in any
language.  Just don't do it.  ;-)

--Johann

--
Johann Hibschman                           johann at physics.berkeley.edu

```