Strange rounding problem

Peter Hansen peter at engcorp.com
Sat Mar 15 13:14:02 EST 2003


Grant Edwards wrote:
> 
> In article <TrCcnZMtctRJ_u6jXTWc3Q at comcast.com>, Marc wrote:
> > So I still don't understand that if I enter a number to be an exact value of
> > .00000096, why it can't be stored as 9.60000000e-007?
> 
> Becausing you entered it in base 10, and it's stored in base 2.  Just
> because it's an exact value in base 10 it doesn't mean it can be represented
> exactly in base 2.

And it might help to envision a base 3 representation, in which
the value 1/3 *could* be represented exactly, but values like 1/2
could not!  In fact, here's a case where a different base than 10
can handle values that base 10 cannot handle...

-Peter




More information about the Python-list mailing list