Floating point subtraction rounding error (NOT display error)

[Keflavich]

On Dec 13, 5:52 pm, Steven D'Aprano <st... at REMOVE-THIS-
cybersource.com.au> wrote:
> On Thu, 13 Dec 2007 14:30:18 -0800, Keflavich wrote:
> > Hey, I have a bit of code that died on a domain error when doing an
> > arcsin, and apparently it's because floating point subtraction is having
> > problems.
>
> I'm not convinced that your diagnosis is correct. Unless you're using
> some weird, uncommon hardware, it's unlikely that a bug in the floating
> point subtraction routines has escaped detection. (Unlikely, but not
> impossible.) Can you tell us what values give you incorrect results?

Here's a better (more complete) example [btw, I'm using ipython]:

In [39]: x = 3.1 + .6
In [40]: y = x - 3.1
In [41]: y == 6
Out[41]: False
In [42]: x == 3.7
Out[42]: True
In [43]: 3.1+.6-3.1 == .6
Out[43]: False
In [45]: (3.1+.6-3.1)/.6
Out[45]: 1.0000000000000002
In [46]: (3.1+.6-3.1)/.6 == 1
Out[46]: False
In [47]: (3.1+.6-3.1)/.6 > 1
Out[47]: True

Therefore, if I try to take the arcsine of that value, instead of
being arcsine(1) = pi, I have arcsine(1.(16 zeroes)1) = math domain
error.  However, see below, I now understand the error.

> > I know about the impossibility of storing floating point
> > numbers precisely, but I was under the impression that the standard used
> > for that last digit would prevent subtraction errors from compounding.
>
> What gave you that impression? Are you referring to guard digits?

I believe that's what I was referring to; I have only skimmed the
topic, haven't gone into a lot of depth

> > Is there a simple solution to this problem, or do I need to run some
> > sort of check at every subtraction to make sure that my float does not
> > deviate?  I'm not sure I know even how to do that.
>
> I still don't quite understand your problem. If you think your float is
> deviating from the correct value, that implies you know what the correct
> value should be. How do you know what the correct value should be?
>
> I should also mention that of course your answer will deviate, due to the
> finite precision of floats.

I'm adding and subtracting things with 1 decimal point; I can do the
math for an given case trivially.  Are you suggesting that I don't
know what the exact floating point quantity should be?

> Obviously not. As you've already shown, the correct value is
> 0.70000000000000018, not 0.69999999999999996 (the closest floating point
> value to 0.7).

The case I cited that you disliked is practically the same as the one
above; sorry I posted one where the value of the variable was
unstated.

> > I was unable to find solutions when searching the web because all of the
> > hits I got were discussing display issues, which I'm not concerned with.
>
> Have you read this?http://docs.sun.com/source/806-3568/ncg_goldberg.html

I started to, but didn't get through the whole thing.  I'm saving that
for bedtime reading after finals.

Anyway, whether or not it's stated in that document, as I presume it
is, the problem I have resulted from a different level of precision
for numbers <1 and numbers >1.  i.e.
1.1000000000000001
 .90000000000000002
so the larger the number (in powers of ten), the further off the
difference between two numbers will be.  I suppose that's why the
"guard digits" did nothing for me - I subtracted a "real number" from
a guard digit.  Still, this seems like very unintuitive behavior; it
would be natural to truncate the subtraction at the last digit of
1.1... rather than including the uncertain digit in the final answer.

> --
> Steven.