[CentralOH] Better Problem Description.

[Neil Ludban]

On Mon, 9 Dec 2013 18:59:01 -0500
Louis Bogdan <looiebwv at gmail.com> wrote:
...
> > On Mon, Dec 9, 2013 at 12:56 PM, Louis Bogdan <looiebwv at gmail.com> wrote:
> >> Now I know that Python does rounding which I don't understand and have
> >> not found any decent, understandable explanation how it does it.  I
> >> understand rounding, even way back when I was proficient with the abacus.

It's really quantization errors, not rounding.  If binary integers
are expressed as a sum of numbers from the set {1, 2, 4, 8, 16, ...}
then binary floating point numbers (from a very simplistic viewpoint)
add to that set the fractions {1/2, 1/4, 1/8, 1/16, ...}.  64-bit
doubles (the typical Python "float" type) enables 53 consecutive
values from that combined set:

http://en.wikipedia.org/wiki/Double_precision_floating-point_format

So numbers like 1, 1+1/2, 1+1/4, 1+1/8, can be represented exactly
in decimal and binary (the minus sign is added here to force the
bin function to output all 64 bits):

--> import struct
--> bin(struct.unpack('Q', struct.pack('d', float('-1.0000')))[0])
'0b1011111111110000000000000000000000000000000000000000000000000000'
--> bin(struct.unpack('Q', struct.pack('d', float('-1.5000')))[0])
'0b1011111111111000000000000000000000000000000000000000000000000000'
--> bin(struct.unpack('Q', struct.pack('d', float('-1.2500')))[0])
'0b1011111111110100000000000000000000000000000000000000000000000000'
--> bin(struct.unpack('Q', struct.pack('d', float('-1.1250')))[0])
'0b1011111111110010000000000000000000000000000000000000000000000000'


1+1/10 works in decimal, but is a repeating pattern in binary:
--> bin(struct.unpack('Q', struct.pack('d', float('-1.1000')))[0])
'0b1011111111110001100110011001100110011001100110011001100110011010'


1+1/3 repeats in both decimal and binary:
--> -4/3.
-1.3333333333333333
--> bin(struct.unpack('Q', struct.pack('d', float(-4/3.)))[0])
'0b1011111111110101010101010101010101010101010101010101010101010101'


And then there's the weird stuff that may concern you:

--> bin(struct.unpack('Q', struct.pack('d', float(-4/3. * 1000 / 1000)))[0])
'0b1011111111110101010101010101010101010101010101010101010101010101'
--> bin(struct.unpack('Q', struct.pack('d', float(-4/3. + 1000 - 1000)))[0])
'0b1011111111110101010101010101010101010101010101010101011000000000'
--> -4/3. + 1000 - 1000
-1.3333333333333712

The result of the last example can vary depending on the compiler
and optimization flags, whether the code runs on the main FPU or
vector coprocessor (eg, MMX), level of IEEE-754 compliance (which
varies from none for early Crays to partly for early NVIDIA GPUs to
full for most modern desktops), and any non-standard options (eg,
rounding mode and denormal truncation) which other parts of your
application or supporting libraries may have set...


Hope that helps.