Strange rounding problem
Grant Edwards
grante at visi.com
Sat Mar 15 18:55:33 CET 2003
In article <TrCcnZMtctRJ_u6jXTWc3Q at comcast.com>, Marc wrote:
> But I didn't make a calculation to achieve this number. I entered it to be
> an exact value. [...]
> By I know that binary digits aren't incapable of holding exact numbers.
Wrong. Partly. Some numbers that are exactly representable in base 10 are
not exactly representble in base 2. For example, the number 0.1 (base 10)
can't be exactly represented in base 2. Just like 0.1 (base 3) can't be
exactly represented as a decimal value in base 10: it's 0.33333 (repeats
forever).
> If that were true, there wouldn't be an accurate computer in the world.
There arent. When dealing with floating point, you must assume that
_no_value_is_exact_.
> So I still don't understand that if I enter a number to be an exact value of
> .00000096, why it can't be stored as 9.60000000e-007?
Becausing you entered it in base 10, and it's stored in base 2. Just
because it's an exact value in base 10 it doesn't mean it can be represented
exactly in base 2.
--
Grant Edwards grante Yow! There's enough money
at here to buy 5000 cans of
visi.com Noodle-Roni!
More information about the Python-list
mailing list