Strange rounding problem

[Grant Edwards]

In article <TrCcnZMtctRJ_u6jXTWc3Q at comcast.com>, Marc wrote:

> But I didn't make a calculation to achieve this number. I entered it to be
> an exact value. [...]
> By I know that binary digits aren't incapable of holding exact numbers.

Wrong. Partly.  Some numbers that are exactly representable in base 10 are
not exactly representble in base 2.  For example, the number 0.1 (base 10)
can't be exactly represented in base 2.  Just like 0.1 (base 3) can't be
exactly represented as a decimal value in base 10: it's 0.33333 (repeats
forever).

> If that were true, there wouldn't be an accurate computer in the world.

There arent.  When dealing with floating point, you must assume that
_no_value_is_exact_.

> So I still don't understand that if I enter a number to be an exact value of
> .00000096, why it can't be stored as 9.60000000e-007?

Becausing you entered it in base 10, and it's stored in base 2.  Just
because it's an exact value in base 10 it doesn't mean it can be represented
exactly in base 2.

-- 
Grant Edwards                   grante             Yow!  There's enough money
                                  at               here to buy 5000 cans of
                               visi.com            Noodle-Roni!