rounding problem

[Dan Bishop]

tom wrote:
> On Wed, 23 Feb 2005 19:04:47 -0600, Andy Leszczynski wrote:
>
> > It is on Windows, Linux, Python 2.3:
> >
> > [GCC 3.3.2 (Mandrake Linux 10.0 3.3.2-6mdk)] on linux2 Type "help",
> > "copyright", "credits" or "license" for more information.
> >  >>> a=1.1
> >  >>> a
> > 1.1000000000000001
...
> > Is it normal?
>
> Yes. Quite normal.
...
> The print statement ,by itself, adds the most minimial formatting,
which
> amounts to dropping the last significant digit your math processor
holds.
> That last digit will *always* contain some arithmetic slop.

Your statement is misleading, because it suggests that your processor
stores digits.  It doesn't; it stores *bits*.

The distinction is important because 1.1 does not have a finite
representation in binary.  Instead, it's the repeating bit sequence 1.0
0011 0011 0011...
This is analogous to 1/3 = 0.333... in base ten.

IEEE 754 requires that all normalized numbers be rounded to 53
significant bits, so "1.1" is actually stored as 0x1.199999999999A,
which is equivalent to the decimal number
1.100000000000000088817841970012523233890533447265625.  Python's repr
function rounds this to 17 significant digits, or "1.1000000000000001".