[Python-ideas] Way to check for floating point "closeness"?

[Andrew Barnert]

On Jan 14, 2015, at 12:04, Ron Adam <ron3200 at gmail.com> wrote:

> What do you think of having a calculator mode or module, which would display specified significant digits in the output, but keep the entire number internally?
> 
> import calc
> calc.fix(4)
> calc.print(epxr)
> 
> And possibly have other general features that can be useful in other more specialised modules.

First, why use a "calc" mode instead of just format('.4')?

Also, this only works when the number of significant digits is small enough to fit in 52 bits; beyond that, it's misleading. Most calculators do something like show 8 decimal digits and store 10 decimal digits, which doesn't have this problem. And in fact, I think in any case where format isn't sufficient, the decimal module is what you need.

(In fact, even with 4 digits it's misleading; two different values can both print identically but there's no function to check that except to stringify them and compare the strings. Calculators that display 8 digits but use 10 for intermediate results usually do comparisons on the 8 digits.)

At any rate, switching to decimal makes it easier to think about and see the rounding errors, and eliminates any additional errors caused by going back and forth to decimal strings (and notice inputs are often decimal strings, with precision represented in decimal significant figures). But ultimately you still have the same problems of tracking the accumulation of error and determining the appropriate relative or absolute (or ulp or log2-relative or whatever) tolerance for equality and inequality comparisons.