[Python-checkins] r73663 - python/branches/py3k/Doc/tutorial/floatingpoint.rst

[raymond.hettinger]

Author: raymond.hettinger
Date: Mon Jun 29 01:21:38 2009
New Revision: 73663

Log:
Clean-up floating point tutorial.

Modified:
   python/branches/py3k/Doc/tutorial/floatingpoint.rst

Modified: python/branches/py3k/Doc/tutorial/floatingpoint.rst
==============================================================================
--- python/branches/py3k/Doc/tutorial/floatingpoint.rst	(original)
+++ python/branches/py3k/Doc/tutorial/floatingpoint.rst	Mon Jun 29 01:21:38 2009
@@ -50,7 +50,7 @@
 
 Stop at any finite number of bits, and you get an approximation.  On most
 machines today, floats are approximated using a binary fraction with
-the numerator using the first 53 bits following the most significant bit and
+the numerator using the first 53 bits starting with the most significant bit and
 with the denominator as a power of two.  In the case of 1/10, the binary fraction
 is ``3602879701896397 / 2 ** 55`` which is close to but not exactly
 equal to the true value of 1/10.
@@ -230,12 +230,8 @@
 and recalling that *J* has exactly 53 bits (is ``>= 2**52`` but ``< 2**53``),
 the best value for *N* is 56::
 
-   >>> 2**52
-   4503599627370496
-   >>> 2**53
-   9007199254740992
-   >>> 2**56/10
-   7205759403792794.0
+    >>> 2**52 <=  2**56 // 10  < 2**53
+    True
 
 That is, 56 is the only value for *N* that leaves *J* with exactly 53 bits.  The
 best possible value for *J* is then that quotient rounded::
@@ -250,14 +246,13 @@
    >>> q+1
    7205759403792794
 
-Therefore the best possible approximation to 1/10 in 754 double precision is
-that over 2\*\*56, or ::
+Therefore the best possible approximation to 1/10 in 754 double precision is::
 
-   7205759403792794 / 72057594037927936
+   7205759403792794 / 2 ** 56
 
 Dividing both the numerator and denominator by two reduces the fraction to::
 
-   3602879701896397 / 36028797018963968
+   3602879701896397 / 2 ** 55
 
 Note that since we rounded up, this is actually a little bit larger than 1/10;
 if we had not rounded up, the quotient would have been a little bit smaller than
@@ -269,24 +264,34 @@
    >>> 0.1 * 2 ** 55
    3602879701896397.0
 
-If we multiply that fraction by 10\*\*60, we can see the value of out to
-60 decimal digits::
+If we multiply that fraction by 10\*\*55, we can see the value out to
+55 decimal digits::
 
-   >>> 3602879701896397 * 10 ** 60 // 2 ** 55
+   >>> 3602879701896397 * 10 ** 55 // 2 ** 55
    1000000000000000055511151231257827021181583404541015625
 
-meaning that the exact number stored in the computer is approximately equal to
-the decimal value 0.100000000000000005551115123125.  Rounding that to 17
-significant digits gives the 0.10000000000000001 that Python displays (well,
-will display on any 754-conforming platform that does best-possible input and
-output conversions in its C library --- yours may not!).
+meaning that the exact number stored in the computer is equal to
+the decimal value 0.1000000000000000055511151231257827021181583404541015625.
+Instead of displaying the full decimal value, many languages (including
+older versions of Python), round the result to 17 significant digits::
+
+   >>> format(0.1, '.17f')
+   '0.10000000000000001'
 
 The :mod:`fractions` and :mod:`decimal` modules make these calculations
 easy::
 
    >>> from decimal import Decimal
    >>> from fractions import Fraction
-   >>> print(Fraction.from_float(0.1))
-   3602879701896397/36028797018963968
-   >>> print(Decimal.from_float(0.1))
-   0.1000000000000000055511151231257827021181583404541015625
+
+   >>> Fraction.from_float(0.1)
+   Fraction(3602879701896397, 36028797018963968)
+
+   >>> (0.1).as_integer_ratio()
+   (3602879701896397, 36028797018963968)
+
+   >>> Decimal.from_float(0.1)
+   Decimal('0.1000000000000000055511151231257827021181583404541015625')
+
+   >>> format(Decimal.from_float(0.1), '.17')
+   '0.10000000000000001'