Strange rounding problem

[Mike Meyer]

mnations at airmail.net (Marc) writes:

> GAP IN ROUTINE 9.6e-007
> GAP BEFORE CONVERSION 9.6e-007
> GAP AFTER CONVERSION 959
> gapList = [[], 9.5999999999999991e-007, '3']
> 
> This appears to be confusing. The variable is set to .00000096 but is
> passed through as 9.5999999999999991e-007. It appears accurate when
> printed before the value is returned, is inaccurate when the full
> return list is printed, accurate when it's printed by itself after the
> return, and then inaccurate again when the division is made. Why does
> it do this, and how can I avoid it?

Actually, all the numbers are the same. What you are seeing is the
difference between str and repr. Repr is used on the elements of a
list when it is printed. But here's what happens when you do things by
hand:

>>> print repr(9.6e-7)
9.5999999999999991e-07
>>> print str(9.6e-7)
9.6e-07
>>> print str(9.5999999999999991e-07)
9.6e-07
>>> 

        <mike
-- 
Mike Meyer <mwm at mired.org>			http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.