[Tutor] range function and floats?

[Steven D'Aprano]

Wayne Werner wrote:
> On Wed, Jan 5, 2011 at 10:43 AM, Steven D'Aprano <steve at pearwood.info>wrote:
> 
>> Wayne Werner wrote:
>>
>>  The decimal module allows you to get rid of those pesky floating point
>>> errors. See http://docs.python.org/library/decimal.html for more info.
>>>
>> That's a myth. Decimal suffers from the same floating point issues as
>> binary floats. It's quite easy to demonstrate the same sort of rounding
>> errors with Decimal as for float:
>>
>>>>> from decimal import Decimal as D
>>>>> x = D(1)/D(3)
>>>>> 3*x == 1
>> False
>>
> 
> I should have clarified - when I say pesky floating point errors I mean
> errors in precision that you naturally would not expect.

Say what?

But that's the whole point -- you *must* expect them, because Decimal 
floating point numbers suffer the *exact* same issues with rounding and 
finite precision as binary floats. It may affect different specific 
numbers, but the issues are exactly the same. Just because Decimal uses 
base 10 doesn't make it immune to the same issues of rounding and 
precision as base 2 floats.


> 1/3 == .333 (repeating forever(?)). But on a computer, you're limited to a
> specific precision point (sort of), and the moment you truncate
> .333(repeating) to *any* finite points of precision you no longer have the
> result of the mathematical operation 1/3. Yes, 1/3*3 == 1, but the error in
> the Decimal module is *only* in division. It might be useful to define a

I'm surprised you can make such an obviously wrong claim -- you even 
discuss errors in addition further down. How can you do that and still 
say that errors only occur with division???

Decimal includes a significant amount of code to detect when the result 
of some operation is inexact, and to signal the user when it occurs. 
This does *not* only happen in division:

 >>> import decimal
 >>> decimal.getcontext().traps[decimal.Inexact] = 1
 >>> a = decimal.Decimal('1e-30')
 >>> b = decimal.Decimal('11.1')
 >>> a+b
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "/usr/local/lib/python3.1/decimal.py", line 1178, in __add__
     ans = ans._fix(context)
   File "/usr/local/lib/python3.1/decimal.py", line 1653, in _fix
     context._raise_error(Inexact)
   File "/usr/local/lib/python3.1/decimal.py", line 3812, in _raise_error
     raise error(explanation)
decimal.Inexact: None



> "repeating" flag in the Decimal module for the results of such operations as
> 1/3, which would get rid of the error in truncation. But this is a

You haven't thought that through very well. How do you deal with square 
roots and other fractional powers without truncation errors?

What happens when you operate on two numbers with this repeating flag -- 
is the Decimal module supposed to include a full-blown algebra system to 
generate exact answers to such things as this?

x = 1/Decimal(3)  # 0.333 repeating
y = 1/Decimal(27)  # 0.037037 repeating
x - y/10**9  # 0.333333333296296296 repeating

How do you deal with numbers like pi or e without truncation?

No, a repeating flag would just add unnecessary complexity for no real 
benefit. If you want to track fractions exactly, use the fractions module.


> fundamentally different error from the standard floating point errors.

But they aren't fundamentally different! That's the point! They are the 
same errors. They effect different numbers, because float is base 2 and 
Decimal is base 10, but they have the same fundamental cause.


> In [106]: 0.9
> Out[106]: 0.90000000000000002
> 
> These are two different numbers, mathematically. If I say x = 0.9, I
> naturally assume that 0.9 is the value in x, not 0.90000000000000002. Of

Yes, and the exact same issue can occur in Decimal. It just affects 
different numbers. 0.9 is exact in base 10, but can't be represented 
exactly in base 2. Fortunately, the opposite is not the case: any finite 
number of binary bits is equivalent to a finite number of decimal 
digits, so unless you overflow the number of digits, you won't get an 
equivalent error for binary->decimal conversion. But there are numbers 
which can't be represented exactly in base 10 at any finite precision. 
1/3 is one of them, so is 5/17 and an infinite number of other examples.

You can even find examples of numbers which are exact in base 2 but 
inexact in Decimal at whatever finite precision you choose.

 >>> 2**120
1329227995784915872903807060280344576
 >>> int( 1/( 1/Decimal(2**120) ) )
1329227995784915872903807060000000000


Choose more digits for Decimal, and 2**120 can be stored exactly, but 
other, larger, numbers can't.


> course it's all in the implementation of floating points, and the fact that
> Python evaluates 0.9 in a floating point context which results in the stored
> value, but that ignores the fact that *naturally* one does not expect this.
> And anyone who's been through 2nd or 3rd grade or whenever they teach about
> equality would expect that this would evaluate to False.

Floating point numbers are not the same as the real numbers you learn 
about in school. It doesn't matter whether you use binary floats or 
decimal floats or base-19 floats.



> In [112]: 0.90000000000000002 == 0.9
> Out[112]: True
> 
> You don't get such silliness with the Decimal module:
> 
> In [125]: D('0.90000000000000002') == D('0.9')
> Out[125]: False


Naturally Decimal is good at representing numbers which can be written 
exactly in decimal. That's what it was designed for :)


> Decimal DOES get rid of floating point errors, except in the case of
> repeating (or prohibitively large precision values)

In other words, decimal gets rid of floating point errors, except for 
floating point errors. That's exactly what floating point errors are: 
numbers which can't be represented exactly in the given number of bits 
or digits.

Regardless of base, you must expect floating point calculations to 
overflow, underflow, or just generally experience rounding errors.

[...]
> If you reset the precision to an incredibly large number:
> decimal.getcontext().prec = 1000

Yes, one advantage of Decimal is that it gives you control of how many 
digits the numbers can store. This is a good, but expensive, feature. 
Nevertheless, it doesn't change the existence of rounding errors. No 
matter what precision you choose, I can ALWAYS give you numbers that 
will overflow or underflow that precision.

> 
> In [131]: x = D(10)**30
> 
> In [132]: x
> Out[132]: Decimal('1000000000000000000000000000000')
> 
> In [133]: x + 100 == x
> Out[133]: False
> 
> Voila, the error has vanished!

But only for that specific value. This is no different from the binary 
world, going from single precision floats to double to quad precision. 
At each stage you can represent more numbers exactly, because you have 
more bits to store them in, but there are always numbers that require 
more. Decimal is no different.


>> So it simply isn't correct to suggest that Decimal doesn't suffer from
>> rounding error.
> 
> 
> I never said rounding errors - I said "pesky floating point errors". When

Which ARE rounding errors. They're *all* rounding errors, caused by the 
same fundamental issue --  the impossibility of representing some 
specific exact number in the finite number of bits, or digits, available.

Only the specific numbers change, not the existence of the errors.


-- 
Steven