Strange rounding problem
grante at visi.com
Sat Mar 15 18:55:33 CET 2003
In article <TrCcnZMtctRJ_u6jXTWc3Q at comcast.com>, Marc wrote:
> But I didn't make a calculation to achieve this number. I entered it to be
> an exact value. [...]
> By I know that binary digits aren't incapable of holding exact numbers.
Wrong. Partly. Some numbers that are exactly representable in base 10 are
not exactly representble in base 2. For example, the number 0.1 (base 10)
can't be exactly represented in base 2. Just like 0.1 (base 3) can't be
exactly represented as a decimal value in base 10: it's 0.33333 (repeats
> If that were true, there wouldn't be an accurate computer in the world.
There arent. When dealing with floating point, you must assume that
> So I still don't understand that if I enter a number to be an exact value of
> .00000096, why it can't be stored as 9.60000000e-007?
Becausing you entered it in base 10, and it's stored in base 2. Just
because it's an exact value in base 10 it doesn't mean it can be represented
exactly in base 2.
Grant Edwards grante Yow! There's enough money
at here to buy 5000 cans of
More information about the Python-list