[Tutor] greater precision?

[eryksun]

On Mon, Oct 29, 2012 at 7:05 AM, Dave Angel <d at davea.name> wrote:
>
> Actually, it's 64 bits.  32 bit fp wouldn't get you anywhere near 18 digits.

A double has 53 bits of precisions, which is 53*log10(2) =~ 15.955
decimal digits. However, one often sees the numbers 15 and 17 quoted
for the precision. It depends. A double is guaranteed to accurately
store a string with 15 decimal digits (round trip). But each 15-digit
decimal string maps to many doubles:

    >>> from struct import unpack

    >>> format(unpack('d', '\x76\x99\x99\x99\x99\x99\xb9?')[0], '.15f')
    '0.100000000000000'
    >>> format(unpack('d', '\xbd\x99\x99\x99\x99\x99\xb9?')[0], '.15f')
    '0.100000000000000'

    >>> 0xbd - 0x76 + 1   # doubles that round to 0.100000000000000
    72

(Note: my Intel processor is little endian, so the least significant
byte is index 0 in the packed double, such as '\x76....'.)

However, to exactly represent each double requires 17 decimal digits:

    >>> format(unpack('d', '\x76\x99\x99\x99\x99\x99\xb9?')[0], '.17f')
    '0.09999999999999951'
    >>> format(unpack('d', '\x77\x99\x99\x99\x99\x99\xb9?')[0], '.17f')
    '0.09999999999999952'

Python says the precision is 15 decimal digits:

    >>> import sys
    >>> sys.float_info.mant_dig   # bits of precision
    53
    >>> sys.float_info.dig        # decimal digits
    15