Float precision and float equality

[sturlamolden]

On 7 Des, 06:43, dbd <d... at ieee.org> wrote:

> If you have
> samples of a sine wave with peak amplitude of one half eps, the "abs(x-
> y) < eps" test would report all values on the sine wave as equal to
> zero. This would not be correct.

You don't understand this at all do you?

If you have a sine wave with an amplitude less than the truncation
error, it will always be approximately equal to zero.

Numerical maths is about approximations, not symbolic equalities.


> 1.0 + eps is the smallest value greater than 1.0, distinguishable from
> 1.0.

Which is the reason 0.5*eps*sin(x) is never distinguishable from 0.


> A constant comparison value is not appropriate.

That require domain specific knowledge. Sometimes we look at a
significant number of digits; sometimes we look at a fixed number of
decimals; sometimes we look at abs(y/x). But there will always be a
truncation error of some sort, and differences less than that is never
significant.






>
> Mark was right, DaveA's discussion explains a strategy to use.
>
> Dale B. Dalrymple