# [PYTHON MATRIX-SIG] floating point equalities

**Paul. Dubois
**
dubois1@llnl.gov

*Tue, 23 Jan 1996 13:59:47 -0800*

Guido is clearly right.
As for how to do approxEqual(a,b), there is no one right answer.
There are two ways of measuring error, relative and absolute.
Relative is usually defined as abs(a-b)/max(abs(a),abs(b)).
Absolute is just abs(a-b).
Suppose the two numbers are 10**-15 and 10**-16. Depending on what you
are doing, the difference between these two is either (a) a factor of
10, which sounds "large", or (b) about 10**-15, which sounds small.
Many people use a "mixed" critera, considering two numbers equal if
their relative difference is small or their absolute difference is
(perhaps a different size of) small. You see tests like:
abs(a-b) < rel_tol * abs(a) + abs_tol
But small is still hard to be precise about, since it really depends on
context. If you get a relative error of 1.e-8 solving a linear system it
might be expected, but that same number might be a sign of trouble in
other contexts where errors more like 1.e-14 are normal.
One solution is say a class RealNumberComparer, which contains
attributes maximum_relative_error, maximum_absolute_error, routines to
set these (with some reasonable defaults set on creation) and an
equals(a,b) to carry out the test.
--
Paul F. Dubois, L-472 (510)-422-5426
Lawrence Livermore National Laboratory FAX (510)-423-9969
Livermore, CA 94550 dubois1@llnl.gov
Consulting: PaulDubois@aol.com
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================