Fwd: Way to check for floating point "closeness"?
oops, forgot ot reply-all ---------- Forwarded message ---------- From: Chris Barker <chris.barker@noaa.gov> Date: Sat, Jan 17, 2015 at 8:38 PM Subject: Re: [Python-ideas] Way to check for floating point "closeness"? To: Terry Reedy <tjreedy@udel.edu> On Fri, Jan 16, 2015 at 8:09 PM, Terry Reedy <tjreedy@udel.edu> wrote:
As near as I can tell, assertAlmostEqual(first, second, places=7, msg=None, delta=None) does one of two possible absolute difference checks (round(first, places) - round(second, places)) == 0.0 abs(first - second) < = delta
There has been discussion of making it more complicated, but I remember that being rejected because other tests require more thought and can be implemented with the above anyway.
anything could be implemented with a simple absolute tolerance method, and for that matter, that is simple enough that there's not much point in having a built in function for it.anted (and a bit harder to write yourself)
assertAlmostEqual((a-b)/d, 0, delta = tol) where d is a, b, and (a+b)/2 as one thinks is appropriate.
while yopu're at it, why not jsut write your own assertion: assert (a-b)/b <= tol that function buys you nothing but potential confusion.
The discussion on the thread seems mostly divorced from the multiple use cases. What do each of a and b represent? Different numbers? or successive approximations of the same number? Are they theoretical 'exact' numbers, approximations of theoretical numbers, or calculations from measurements with error? If the latter, how big is the error? And why are we asking anyway? We are usually asking 'close enough' for some purpose, but purposes differ.
yes, they sure do -- but anything in the standard library would be for general use, where the details don't matter too much -- if you are doing something really specific, write your own test.
(Someone claimed that 'nothing is close to zero'. This is nonsensical both in applied math and everyday life.)
I'm pretty sure someone (more than one of use) asserted that "nothing is *relatively* close to zero -- very different. And indeed, that is still a primary stumbling block (at least for me) to putting a relative tolerance function into the stdlib: it will work fine in most cases, and be easy to understand and reason about, but it will not work if one of the input values is zero, and casual users may get tripped up by that. But at this point, i think that just needs note in the docs -- after all, they are likely to get tripped up by that if they write the code themselves anyway. And I really wanted a way to have a default behavior that would do a reasonable transition to an absolute tolerance near zero, but I no longer thing that's possible. (numpy's implimentaion kind of does that, but it is really wrong for small numbers, and if you made the default min_tolerance the smallest possible representable number, it really wouldn't be useful. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On 01/17/2015 11:37 PM, Chris Barker wrote:
(Someone claimed that 'nothing is close to zero'. This is nonsensical both in applied math and everyday life.)
I'm pretty sure someone (more than one of use) asserted that "nothing is *relatively* close to zero -- very different.
Yes, that is the case.
And I really wanted a way to have a default behavior that would do a reasonable transition to an absolute tolerance near zero, but I no longer thing that's possible. (numpy's implimentaion kind of does that, but it is really wrong for small numbers, and if you made the default min_tolerance the smallest possible representable number, it really wouldn't be useful.
I'm going to try to summarise what I got out of this discussion. Maybe it will help bring some focus to the topic. I think there are two case's to consider. # The most common case. rel_is_good(actual, expected, delta) # value +- %delta. # Testing for possible equivalence? rel_is_close(value1, value2, delta) # %delta close to each other. I don't think they are quite the same thing. rel_is_good(9, 10, .1) --> True rel_is_good(10, 9, .1) --> False rel_is_close(9, 10, .1) --> True rel_is_close(10, 9, .1) --> True In the "is close" case, it shouldn't matter what order the arguments are given. The delta is the distance from the larger number the smaller number is. (of the same sign) So when calculating the relative error from two values, you want it to be consistent with the rel_is_close function. rel_is_close(a, b, delta) <---> rel_err(a, b) <= delta And you should not use the rel_err function in the rel_is_good function. The next issue is, where does the numeric accuracy of the data, significant digits, and the languages accuracy (ULPs), come into the picture. My intuition.. I need to test the idea to make a firmer claim.. is that in the case of is_good, you want to exclude the uncertain parts, but with is_close, you want to include the uncertain parts. Two values "are close" if you can't tell one from the other with certainty. The is_close range includes any uncertainty. A value is good if it's within a range with certainty. And this excludes any uncertainty. This is where taking in consideration of an absolute delta comes in. The minimum range for both is the uncertainty of the data. But is_close and is_good do different things with it. Of course all of this only applies if you agree with these definitions of is_close, and is_good. ;) Cheers, Ron
On Sun, Jan 18, 2015 at 11:27 AM, Ron Adam <ron3200@gmail.com> wrote: I'm going to try to summarise what I got out of this discussion. Maybe it
will help bring some focus to the topic.
I think there are two case's to consider.
# The most common case.
why do you think this is the most common case?
rel_is_good(actual, expected, delta) # value +- %delta.
# Testing for possible equivalence? rel_is_close(value1, value2, delta) # %delta close to each other.
I don't think they are quite the same thing.
rel_is_good(9, 10, .1) --> True rel_is_good(10, 9, .1) --> False
rel_is_close(9, 10, .1) --> True rel_is_close(10, 9, .1) --> True
agreed -- they are not the same thing. But I'm not convinced that they are all that different from a practical perspective -- 0.1 is a very large relative tolerance (I wouldn't cal it delta for a relative measure) if you use a more-common, much smaller tolerance (1e-8 -- 1e-12 maybe?) then the difference between these becomes pretty minimal. And for the most part, this kind of testing is looking for an "approximation" -- so you can't get really upset about exactly where the cut-off is. Though, given my thoughts on that, I suppose if other people want to be able to clearly specify which of the two values should be used to scale "relative", then it won't make much difference anyway. The next issue is, where does the numeric accuracy of the data, significant
digits, and the languages accuracy (ULPs), come into the picture.
My intuition.. I need to test the idea to make a firmer claim.. is that in the case of is_good, you want to exclude the uncertain parts, but with is_close, you want to include the uncertain parts.
I think this is pretty irrelevant -- you can't do better than the precession of the data type, doesn't make a difference which definition you are using -- they only change which value is used to scale relative. There are cases where ULPs, etc are key -- those are for testing precision of algorithms, etc, and I think a special use case that would require a different function -- no need to try to cram it all into one function.
Two values "are close" if you can't tell one from the other with certainty. The is_close range includes any uncertainty.
I think "uncertainly" is entirely use-case dependent -- floating point calculations are not uncertain -- with a defined rounding procedure, etc, they are completely deterministic. IT can often make things easier to think about if you think of the errors as random, but they are, in fact, not random. And if you care about the precision down to close to limits of representation, then you care about ULPS, etc, and you'd better be reasoning about the errors carefully. This is where taking in consideration of an absolute delta comes in. The
minimum range for both is the uncertainty of the data. But is_close and is_good do different things with it.
an absolute delta is simply a different use-case -- sometimes you know exactly how much difference you care about, and sometimes you only care that the numbers are relatively close (or, in any case, you need to test a wide variety of magnitudes of numbers, so don't want to have to calculate the absolute delta for each one). This is akin to saying that the numbers are the same to a certain number of significant figures -- a common and usefull way to think about it. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Jan 18, 2015, at 12:14, Chris Barker <chris.barker@noaa.gov> wrote:
There are cases where ULPs, etc are key -- those are for testing precision of algorithms, etc, and I think a special use case that would require a different function -- no need to try to cram it all into one function.
I think the right answer there is to add a float_difference that follows the same rules as the IEEE/C/POSIX nextafter function and friends. (And also maybe to add nextafter and friends--note that decimal already has similar functions, but named with underscores.) With that, it's trivial to add ulps-based tests; without it, it's very hard. And then ulps can be removed from the discussion on absolute and relative error.
On Jan 18, 2015, at 1:02 PM, Andrew Barnert <abarnert@yahoo.com> wrote:.
I think the right answer there is to add a float_difference that follows the same rules as the IEEE/C/POSIX nextafter function and friends. (And also maybe to add nextafter and friends--note that decimal already has similar functions, but named with underscores.) With that, it's trivial to add ulps-based tests; without it, it's very hard.
And then ulps can be removed from the discussion on absolute and relative error.
Sounds good to me. -CHB
participants (4)
-
Andrew Barnert -
Chris Barker -
Chris Barker - NOAA Federal
-
Ron Adam