Floating point errors?
David Lees
debl at nospammytheworld.com
Fri May 11 02:43:28 CEST 2001
John,
You are correct that my terminology is wrong (too many years since 9th
grade). The problem is that 3/10, when divided out in base 10 has a
repeating decimal value of zero. However, when you do the division base
2, the repeating decimal is non-zero, so you end up having to truncate
at however many bits go into the floating point mantissa.
david lees
"John J. Lee" wrote:
>
> On Thu, 10 May 2001, David Lees wrote:
>
> > Complexity has nothing to do with rounding. Try:
> >
> > >>> 2.3-2.0
> > 0.29999999999999982
> >
> >
> > Rounding is exactly the issue and this looks like the old classic
> > problem, namely rational numbers in one base (say 10) can be irrational
> > in another base (say 2)
>
> Rationality is a property of numbers, not their representation. 0.3 is a
> rational number because you can represent it as a ratio of two integers,
> which is as true in binary as it is in decimal. I know what you mean,
> though.
>
> John
More information about the Python-list
mailing list