On Sun, Jan 18, 2015 at 11:51 PM, Stephen J. Turnbull wrote:

...

But I still think we can provide something that is useful for most use-cases, and would like to propose what that is, and what the decision points are:

It would be useful, that is, if the user knows what she is doing. The problem exposed by the plethora of use cases is that in many cases users don't know what they're doing, and it's *not* just a matter of forgetting to take the absolute value.

I was afraid of this -- my take is that EVERY algorithm has corner cases, an special cases, and places where it is appropriate and where it is not. And this list (and all lists...) is full of folks that like to hash this stuff out. So the fact that there has been a long and drawn out discussion does not mean that there is not a generally useful function to be had. And I think, in fact, that such a function does is exist that will work most of teh time, even for users that have not thought it out carefully. We, in fact have a lot of evidence that this is needed: * numpy has close() and all_close -- they are widely used, mostly successfully * Steven took that time to write one (and tests for it) for the statistics module tests. * The unittest module has something -- and a poor one (IMHO) at that -- but is still useful. * Boost has one (not Python, but still....)

It will not do a simple absolute comparison -- that is the job of a

...

different function, or, better yet, folks just write it themselves:

abs(x - y) <= delta

really isn't much harder to write than a function call:

absolute_diff(x,y,delta)

for relative error

abs(x - y)/min(abs(x), abs(y))

really isn't that hard to write -- if you know that's what you want.

that is, I think, just enough harder to think about and write than the absolute comparison method, that it's worth it. And if others think an absolute_close is worth adding, too, I have no problem with that. But Neil and Ron *don't* want that: they know which value is correct

and that it's never zero.

I think my proposal satisfies both Neil and Ron's use case -- or a version of it can, and also Steven's.

...

It will be designed for floating point numbers, and handle inf, -inf, and NaN "properly".

Which means what? Something different from what a conforming IEEE 754 implementation would do?

no, sorry for being lazy in writing, -- I meant it would conform to IEEE 754. i.e. NaN would not be close to anything, including another NaN, and inf and -inf would only be close to themselves. (though I don't know that ieee ever mentions non-exact comparison...) The farther we get into this, the more handwaving is being done by

advocates. Despite what I wrote above, it's not trivial to write a correct symmetric relative error: some such function would be useful.

if it's not trivial, then all the reason to to put it in the standard lib.

But I think that it should have a name that (at least to some extent) indicates what it actually does (unlike is_close(), which users are likely to assume means what they think it means because they haven't thought about the alternatives).

Indeed -- I think that the name of the function, and it's parameters need to be carefully chosen to best convey what they mean. (and the docstring). What I"d like to do is determine whether there is any hope of coming to a consensus about what a reasonable implementation would look like. If not, then so be it. If so, then I guess I"ll need to write a PEP and then hash out the naming and details. So I think the next step is to hear form the active participants in this thread as to whether they think there's hope, and/or from Guido and other core developers as to whether they're ready to reject it at this point -- I don't want to waste any more of my or anyone else's time. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On Mon, Jan 19, 2015 at 12:22 PM, Ethan Furman wrote:

I think it's worth having, but it will definitely need a PEP.

Fair enough -- thanks. What's the procedure? Do I work up a draft, publish it somewhere temporary, and start hashing it out on this list? or or do I put the draft up with the official PEPs first? -Chris Now to find the time to write it...... -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On 1/19/2015 2:54 PM, Chris Barker wrote:

But I think that it should have a name that (at least to some extent) indicates what it actually does (unlike is_close(), which users are likely to assume means what they think it means because they haven't thought about the alternatives).

Indeed -- I think that the name of the function, and it's parameters need to be carefully chosen to best convey what they mean. (and the docstring).

If the function only handles 'relatively close', I would be happier with 'relative_close' or 'rel_close' than 'is close' since the latter tends to imply to me absolute closeness. The proposed doc (and docstring) should mention absolute tolerance and 'abs(a-b) < tol' as an alternative, expecially near or at 0. -- Terry Jan Reedy

On Jan 19, 2015, at 12:28, Chris Barker wrote:

Has anyone in this thread, or in any test code you've seen or written, provided a tolerance with more than one significant figure? i.e, I always use 1e-12 or 1e-10, I've never used a tolerance anything like 0.000153427213

If you're working with measurement error or mechanical tolerance or something similar, you often have something with a non-unit mantissa. For example, the first resistor in the examples for the standard color code chart is 4.7KOhm +/- 10%, so you've got an error of 4.7e2. Also, keep in mind that if you do an error propagation analysis, you're unlikely to get round numbers. Even for a dead-simple case, where you start off with two non-covariant values that are +/- 1.00e-5 and add them, the result is +/- 1.41e-5. The reasons you often only have one significant figure are (a) you're hand-waving the error analysis, or (b) your error ends up a couple orders of magnitude smaller than your values and you've only got a couple significant digits. (1.234e5 +/- 6.789e0 is no more meaningful than 1.234e5 +/- 1e1, after all.)

On 01/19/2015 02:28 PM, Chris Barker wrote:

On Mon, Jan 19, 2015 at 11:51 AM, Ron Adam mailto:ron3200@gmail.com> wrote:

Here is a gist with a sample implementation:

https://gist.github.com/__PythonCHB/6e9ef7732a9074d9337a https://gist.github.com/PythonCHB/6e9ef7732a9074d9337a

For the most part I think it looks good.

Boost describes both a week and strong version, but I didn't see why they choose the strong version.

Actually, Boost didn't choose, there is a flag you can set to select which one to use. I don't want to do that, if you really want something special, write it yourself. I did choose the "strong" version -- it just seemed more conservative. Also it's what I think Steven chose for his version (though not with and, but the result is the same)

The two different cases probably should be two different functions, and not use a flag. I'm not suggesting we need both.

As, in the common case, the tolerance is approximate, and usually small, then it doesn't matter which version we use: string, weak, or declared which value to scale to. But I prefer a symmetric version, as I suspect hat will be the least surprising -- it's good to get the same answer every time, even if it is approximate!

Well, approximate-in, approximate-out. Unfortunately that applies to all math. If you use approximations in your math, you will get approximate answers. But many computer programmers like things to be a bit more precise. So I suggest not using the word approximate or estimate in the docs. The calculation isn't an approximation even if the values you supply it are. It actually is a well defined range test.

I hope we can come to some consensus that something like this is the way to go.

Good examples will help with this. It may also help with choosing a good name.

you mean use-case examples? rather than specific value examples?

Yes, specific values don't indicate how something should be used.

To me, the strong version is an "is-good" test, and the weak version is an "is-close" test. I think it could be important to some people.

I like the idea of being able to use these as a teaching tool to demonstrate how our ideas of closeness, equality, and inequality can be subjective.

Are you suggesting that we allow a flag for the user to set to choose whether ot use weak or string version? I'd rather not -- I see this is a practical, works most of the time thing, not a teaching tool, or a "provides every use case" tool.

No flag, just that it needs to be well defined and not mix explanations of use of one with the other. Pick one, and then document how to use it correctly. At some point maybe someone will add the other if it's needed. It is possible to use one for the other if you take the differences into account in the arguments.

There are two cases...

1: (The weak version is require for this to work.)

Two numbers are definitely not equivalent if they are further apart than the largest error amount. (The larger number better indicates the largeness of the the possible relative error.)

And two numbers are close if you can't determine if they are equivalent, or not-equivalent with certainty.*

(* "close numbers" may include equivalent numbers if you define it as a set of all definitely not-equivalent numbers.)

2: (The strong version is required for this to work.)

A value is good if it's within a valid range with certainty. It is less than the smaller relative range of either number. The smaller number better indicates the magnitude of smallness.

So case 1 should be used to test for errors, and case 2 should be used to test for valid ranges.

It seems you have the 2nd case in mind, and that's fine. Some of us where thinking of the first case, and possibly switching from one to the other during the discussion which is probably why it got confusing or repetitious at some points.

yes, I suppose I do -- and again, in the common use case, where the tolerance is also approximate, it really doesn't matter.

I'm curious to what degree it can matter, given different size values and tolerances?

I think both of these are useful, but you definitely need to be clear which one you are implementing, and to document it clearly.

yup.

Cheers, Ron

If you decide to make a PEP, please list the other algorithms you found and their definitions. Personally, I'm for being consistent with numpy and defining math.isclose similar to numpy.isclose for consistency alone. If you decide to invent a relative error function, my suggestion is: (a-b)/b + log(b/a), which is nonnegative, zero only at equality, and otherwise penalizes positive a for being different than some target positive b. To me, it seems like guessing b using 1.9b is better than guessing it as 0.1b, and so on. This corresponds to exponential KL divergence, which has a clear statistical meaning, but only applies to positive numbers. Best, Neil On Monday, January 19, 2015 at 7:59:34 PM UTC-5, Chris Barker - NOAA Federal wrote:

...

On Jan 19, 2015, at 3:17 PM, Ron Adam javascript:> wrote: The two different cases probably should be two different functions, and not use a flag. I'm not suggesting we need both.

I agree there--"strong" it is for the initial proposal at least.

...

Well, approximate-in, approximate-out. Unfortunately that applies to all math

...

But many computer programmers like things to be a bit more precise.

Well, yes and no . Floating point has its limitations, and many programmers simply use double (python float), and hope the results are good enough. What I'm hoping is that this will at least make it more likely folks will apply the "good enough" criteria.

Proper floating point error analysis is hard, most of us leave it for the experts, and I'm not suggesting this is useful for them.

...

So I suggest not using the word approximate or estimate in the docs.

The calculation isn't an approximation even if the values you supply it are. It actually is a well defined range test.

...

...

I hope we can come to some consensus that something like this is the way to go.

Good examples will help with this. It may also help with choosing a good name.

you mean use-case examples? rather than specific value examples?

Yes, specific values don't indicate how something should be used.

...

To me, the strong version is an "is-good" test, and the weak version

...

...

an "is-close" test. I think it could be important to some people.

I like the idea of being able to use these as a teaching tool to demonstrate how our ideas of closeness, equality, and inequality can be subjective.

Are you suggesting that we allow a flag for the user to set to choose whether ot use weak or string version? I'd rather not -- I see this is a practical, works most of the time thing, not a teaching tool, or a "provides every use case" tool.

No flag, just that it needs to be well defined and not mix explanations of use of one with the other. Pick one, and then document how to use it correctly. At some point maybe someone will add the other if it's needed.

It is possible to use one for the other if you take the differences into account in the arguments.

...

There are two cases...

1: (The weak version is require for this to work.)

Two numbers are definitely not equivalent if they are further apart than the largest error amount. (The larger number better indicates

is the

...

...

largeness of the the possible relative error.)

And two numbers are close if you can't determine if they are equivalent, or not-equivalent with certainty.*

(* "close numbers" may include equivalent numbers if you define it as a set of all definitely not-equivalent numbers.)

2: (The strong version is required for this to work.)

A value is good if it's within a valid range with certainty. It is less than the smaller relative range of either number. The smaller number better indicates the magnitude of smallness.

So case 1 should be used to test for errors, and case 2 should be used to test for valid ranges.

It seems you have the 2nd case in mind, and that's fine. Some of us where thinking of the first case, and possibly switching from one to the other during the discussion which is probably why it got confusing or repetitious at some points.

yes, I suppose I do -- and again, in the common use case, where the tolerance is also approximate, it really doesn't matter.

I'm curious to what degree it can matter, given different size values and tolerances?

...

I think both of these are useful, but you definitely need to be clear which one you are implementing, and to document it clearly.

yup.

Cheers, Ron

_______________________________________________ Python-ideas mailing list Python...@python.org javascript: https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

---

Python-ideas mailing list Python...@python.org javascript: https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On Mon, Jan 19, 2015 at 08:10:35PM -0800, Neil Girdhar wrote:

If you decide to invent a relative error function,

The error functions we have been talking about are hardly "invented". They're mathematically simple and obvious, and can be found in just about any undergraduate book on numerical computing: - The absolute error between two quantities a and b is the absolute difference between them, abs(a-b). - The relative error is the difference relative to some denominator d, abs(a-b)/abs(d), typically with d=a or d=b. If you happen to know that b is the correct value, then it is common to choose b as the denominator. If you have no a priori reason to think either a or b is correct, or if you prefer a symmetrical function, a common choice is to use d = min(abs(a), abs(b)). See, for example: http://mathworld.wolfram.com/AbsoluteError.html http://mathworld.wolfram.com/RelativeError.html

my suggestion is: (a-b)/b + log(b/a), which is nonnegative, zero only at equality, and otherwise penalizes positive a for being different than some target positive b. To me, it seems like guessing b using 1.9b is better than guessing it as 0.1b, and so on. This corresponds to exponential KL divergence, which has a clear statistical meaning, but only applies to positive numbers.

Do you have a reference or derivation for this? I'm happy to admit that I'm no Knuth or Kahan, but I've read a bit of numerical computing[1] and I've never seen anyone add a log term. I'm not even sure why you would do so. [1] I know just enough to know how much I don't know. -- Steve

On Tue, Jan 20, 2015 at 2:58 AM, Neil Girdhar wrote:

I'm not against this. What I meant by invention was that it appears that no one defines relative error this way that I could see, which is why I suggested you collect some examples.

Did you look at the Boost docs? That's pretty much what they do, and it was apparently derived from Kahan -- can't get a better pedigree than that ;-). To some extent, whether it's symmetric or not depends on what you name it. to me, at least "is_close" implies symmetry -- a is close to be , but b is not close to a? pretty weird. but if you call it is_close_to, then I could see that a is close to be, but b may not be close to a (though that still seems pretty odd). ANyone got a better name? Again, I'm pretty sure that the most common use cases (and the one where folks are likely to use a stdlib function, rather than write their own) are where the tolerance is approximate anyway -- i.e. ten decimal digits or so (or 1e-10) -- these distinctions are irrelevant in that case, but the symmetric versions may make for fewer surprises. The mathworld article, numpy, and wikipedia all choose the asymmetric

relative error, which is what I prefer as well.

See, for example:

...

http://mathworld.wolfram.com/AbsoluteError.html http://mathworld.wolfram.com/RelativeError.html

thanks for the links. I think I'm sold on the relative error as it's normally defined, but I'd

...

rather we see what all of the other packages have done and why. I don't think the default should be symmetrical since to me the asymmetrical case seems more common.

How about:

relative_error(true_value, approximate_value, absolute_tolerance=..., relative_tolerance=..., tolerance_relative_to=None)

Interesting -- numpy, for instance puts the "true value" (the one used to scale) second in the parameter list... one reason that I like symmetric. But using clear names like that does make it more clear that it isn't symmetric.

if tolerance_relative_to is None, replace it with true_value (or min(true, approx) if we go with the symmetrical definition).

hmm -- I'm a bit wary of too many options, but if we can't decide on the "best" way, then that may be the way to go -- as long as the defaults are reasonable. [1] I know just enough to know how much I don't know.

me too -- I _barely_ passed a class with Kahan, which really did teach me far more about what I don't know than anything else! -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

While we are at this, since someone will have to step up to come up with a PEP, I have these two ideas to drop in the brainstorm - as I'd like to the one writting the PEP text to at least have them in mind: 1) Implement a Number class that behaves as a float number, and does the fuzzy comparisons automatically. Justificative: in a whole lot of code, having a "casted" FuzzyFloat(mynumber) to be able to be compared to others with "==" and "<=" would be much more readable than the Javaesque " if is_close(my_number, other) or my_number < other: " (against: "if FuzzyFloat(my_number) <= other :" , or simply "if my_number <= other: " if we are already working with the FuzzyFloat type) It could use a "context" just like "decimal" to specify comparison parameters I made a naive approach to this when these threads started a couple of days ago - of course it is far from a real implementation - https://gist.github.com/jsbueno/1e62be882639a13180f6 (forgive me for not using ABC's there, I just wrote that to have a "feel" on this idea on the interactive console - and it was a nice test-drive) 2) Put a though on how things could work with other numeric types than float and complex - "decimal" comes to mind. js -><-

On 21 January 2015 at 09:59, Steven D'Aprano wrote:

On Wed, Jan 21, 2015 at 09:27:39AM -0200, Joao S. O. Bueno wrote:

...

1) Implement a Number class that behaves as a float number, and does the fuzzy comparisons automatically.

-1 on this.

...

Justificative: in a whole lot of code, having a "casted" FuzzyFloat(mynumber) to be able to be compared to others with "==" and "<=" would be much more readable than the Javaesque

" if is_close(my_number, other) or my_number < other: " (against: "if FuzzyFloat(my_number) <= other :" , or simply

"if my_number <= other: "

If that sort of comparison is important, a helper function is easy to write. But it really sounds like you want a full interval arithmetic class, which is a lot more than just fuzzy comparisons.

There are two problems with overriding == in this way:

(1) What do you mean by "is close"? Do you mean that they are within 0.0001 or 0.00000001 or within 10000.0? With a binary operator == there is no easy way to specify an error tolerance, which means you're stuck with using a language-wide default which is very unlikely to be suitable for your application. A context manager is one solution, but that's still awkward. Think about doing:

a == b or x == y

where the a, b comparison and x, y comparison have different tolerances.

Well - that would be treated by the fuzzyfloat context -just in the same way contexts are used by decimal.Decimals And of course, if one happens to be comparing numbers with different tolerances, the smaller should rule. (Although when comparing with a non-fuzzy, which has a tolerance of 0, the fuzzy one should be used or there would be no point in having it in the first place. So, yeah, there is a problem there. Requiring both numbers to be Fuzzy might be a way out)

(2) Fuzzy equality means that there are situations where:

a == b and b == c

but

a != c

The language APL tried to turn this into a feature:

The intransitivity of [tolerant] equality is well known in practical situations and can be easily demonstrated by sawing several pieces of wood of equal length. In one case, use the first piece to measure subsequent lengths; in the second case, use the last piece cut to measure the next. Compare the lengths of the two final pieces. — Richard Lathwell, APL Comparison Tolerance, APL76, 1976

Point taken. Anyway, upon reminding how unconfortable ".is_equal" is in source code, I think having an explicit class that can use "==" in this way is not that bad;

but I'm not so sure. If we introduced a new comparison operator (perhaps the Unicode symbol ≈ or the ASCII ~= ?) I'd be less concerned but I think overriding == for fuzzy equality is a mistake.

That is nice. As I said, I'd like people to at least having this in mind.

-- Steve ______________________

It just hit me that while this intransivity is strange, mathematically speaking, it _is_ indeed what happens in "the real world" when we are dealing with measurements - this wood piece example being more than enough to show it.

The wood piece example is analogous to accumulated floating point error. A good carpenter knows not to do that, just as a good programmer knows not to successively add 0.1 floats and expect to get a nice round result. But measurement of error is a key issue here. For the use cases Ron brings up, that is, answering the question " is this measured value within some relative tolerance of another, known, and presumed to be exact, value?" Then yes, the asymmetric approach, where you specify which value is the "known" one, is the way to do it. But is that the use-case that's likely to be common? I think there are a heck of a lot of people that need to check if two values are close to each other for tests and whatnot, than writing python code for a quality assurance system. So what do we need for the common use case? It may not be that different, actually. You probably do often have a computed value that you are checking against a known result. In which case the asymmetric approach is clear and makes sense. Again, for what I think is the MOST common case: " I just want to know if I'm in the ballpark" it doesn't matter which is used. But we should probably call it "is_close_to" or some such, and name the parameters clearly. -Chris

On 1/21/2015 6:59 AM, Steven D'Aprano wrote:

On Wed, Jan 21, 2015 at 09:27:39AM -0200, Joao S. O. Bueno wrote:

...

1) Implement a Number class that behaves as a float number, and does the fuzzy comparisons automatically.

-1 on this.

(2) Fuzzy equality means that there are situations where:

a == b and b == c

but

a != c

The language APL tried to turn this into a feature:

The intransitivity of [tolerant] equality is well known in practical situations and can be easily demonstrated by sawing several pieces of wood of equal length. In one case, use the first piece to measure subsequent lengths; in the second case, use the last piece cut to measure the next. Compare the lengths of the two final pieces. — Richard Lathwell, APL Comparison Tolerance, APL76, 1976

but I'm not so sure. If we introduced a new comparison operator (perhaps the Unicode symbol ≈ or the ASCII ~= ?) I'd be less concerned but I think overriding == for fuzzy equality is a mistake.

The proper operation of sets (and dicts) depends on == being transitive. Letting D0 = Decimal(0), we once had 0.0 == 0 and 0 == D0 but 0.0 != D0 (now fixed) and the result was obscure bugs. One of the design criteria for, I believe, Enums was that == be transitive. -- Terry Jan Reedy

On Wed, Jan 21, 2015 at 11:27 AM, Joao S. O. Bueno wrote:

1) Implement a Number class that behaves as a float number, and does the fuzzy comparisons automatically.

Justificative: in a whole lot of code, having a "casted" FuzzyFloat(mynumber) to be able to be compared to others with "==" and "<=" would be much more readable than the Javaesque

" if is_close(my_number, other) or my_number < other: " (against: "if FuzzyFloat(my_number) <= other :" , or simply

I'm sure this has come up somewhere, but I have to say that I've never in my life seen a situation where an approximate <= was the correct solution to a problem. I've seen lots of exact <= (which you just assume is interchangeable with <) and approximate ==, but never approximate <=. -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org

On Tue, Jan 20, 2015 at 8:43 PM, Ron Adam wrote:

The more we talk about this, the more I'm beginning to dislike the symmetric version.

We are trading an explicit (first, second) relationship with an implicit (smaller, larger) relationship. For Python's general use, I don't like that. Sorry. :/

I don't think there's really anything more "implicit" about it. And you could use the mean of a and b if you wanted it a bit less "implicit". But your other point is well taken, if you DO have a clearly defined "correct" value, then that is the one to scale the error on. And it's a good idea to provide that for those cases. Care to help me write the PEP ? -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On 01/20/2015 09:32 AM, Chris Barker wrote:

On Mon, Jan 19, 2015 at 8:10 PM, Neil Girdhar wrote:

...

If you decide to make a PEP, please list the other algorithms you found and their definitions.

sure -- care to contribute ;-)

...

Personally, I'm for being consistent with numpy and defining math.isclose similar to numpy.isclose for consistency alone.

This is a good question for the larger, non-numpy group. I'm personally a heavy numpy user, and don't see any reason to be consistent -- and I (and others on this list) have some real issues with the numpy approach. But we shouldn't call it isclose() if it's not the same algorithm...

I'm not a numpy nor heavy maths user, and see no reason why we should try to match numpy -- if you want/need numpy, get numpy. There is also no reason to not use the same name, even if it is not the same algorithm -- if the answer we're looking for is "is it close?" then is_close is a fine name (or isclose, similar to the other is... functions and methods Python already has). I think for simplicity' sake, a symmetric function would be better. -- ~Ethan~

On Mon, Jan 19, 2015 at 8:29 PM, Neil Girdhar wrote:

Also for complex numbers, I think comparing the magnitude (distance from the origin, or absolute value) of (x-y) to the size of x or y makes more sense than calling is_close on the real and imaginary parts. What if the real parts are much larger than the imaginary parts, e.g. x=1e5+1e-5j, y=1e5-1e-5j. Do you think x and y are not close?

I've vacillated on this one. Personally I have no use case in mind, so really hard to know what's best. but your example case was exactly what I was thinking -- if the magnitude of one component is very different that the other, then I'm not sure if you would want it to swamp the answer, and requiring both components to be "close" would be the more conservative approach. On the other hand, there are cases where the exact result of a computation on complex numbers is, in fact, a pure real or imaginary number. In these cases, the computed result may be a large value in one component and a tiny value in the other, and you would test against the "actual" value, which would have a zero in there, so the relative error of that component by itself would never be "small". So yes, probably better to use the absolute value(s) to scale relative. But if anyone has some use-cases that would suggest the more strict approach, speak now. -Chris

Best,

Neil

On Monday, January 19, 2015 at 1:33:44 AM UTC-5, Chris Barker wrote:

...

OK folks,

There has been a lot of chatter about this, which I think has served to provide some clarity, at least to me. However, I'm concerned that the upshot, at least for folks not deep into the discussion, will be: clearly there are too many use-case specific details to put any one thing in the std lib. But I still think we can provide something that is useful for most use-cases, and would like to propose what that is, and what the decision points are:

A function for the math module, called somethign like "is_close", "approx_equal", etc. It will compute a relative tolerance, with a default maybe around 1-e12, with the user able to specify the tolerance they want.

Optionally, the user can specify an "minimum absolute tolerance", it will default to zero, but can be set so that comparisons to zero can be handled gracefully.

The relative tolerance will be computed from the smallest of the two input values, so as to get symmetry : is_close(a,b) == is_close(b,a). (this is the Boost "strong" definition, and what is used by Steven D'Aprano's code in the statistics test module)

Alternatively, the relative error could be computed against a particular one of the input values (the second one?). This would be asymmetric, but be more clear exactly how "relative" is defined, and be closer to what people may expect when using it as a "actual vs expected" test. --- "expected" would be the scaling value. If the tolerance is small, it makes very little difference anyway, so I'm happy with whatever consensus moves us to. Note that if we go this way, then the parameter names should make it at least a little more clear -- maybe "actual" and "expected", rather than x and y or a and b or... and the function name should be something like is_close_to, rather than just is_close.

It will be designed for floating point numbers, and handle inf, -inf, and NaN "properly". But is will also work with other numeric types, to the extent that duck typing "just works" (i.e. division and comparisons all work).

complex numbers will be handled by: is_close(x.real, y.real) and is_close(x.imag, y.imag) (but i haven't written any code for that yet)

It will not do a simple absolute comparison -- that is the job of a different function, or, better yet, folks just write it themselves:

abs(x - y) <= delta

really isn't much harder to write than a function call:

absolute_diff(x,y,delta)

Here is a gist with a sample implementation:

https://gist.github.com/PythonCHB/6e9ef7732a9074d9337a

I need to add more tests, and make the test proper unit tests, but it's a start.

I also need to see how it does with other data types than float -- hopefully, it will "just work" with the core set.

I hope we can come to some consensus that something like this is the way to go.

-Chris

On Sun, Jan 18, 2015 at 11:27 AM, Ron Adam wrote:

...

On 01/17/2015 11:37 PM, Chris Barker wrote:

...

(Someone claimed that 'nothing is close to zero'. This is nonsensical both in applied math and everyday life.)

I'm pretty sure someone (more than one of use) asserted that "nothing is *relatively* close to zero -- very different.

Yes, that is the case.

And I really wanted a way to have a default behavior that would do a

...

reasonable transition to an absolute tolerance near zero, but I no longer thing that's possible. (numpy's implimentaion kind of does that, but it is really wrong for small numbers, and if you made the default min_tolerance the smallest possible representable number, it really wouldn't be useful.

I'm going to try to summarise what I got out of this discussion. Maybe it will help bring some focus to the topic.

I think there are two case's to consider.

# The most common case. rel_is_good(actual, expected, delta) # value +- %delta.

# Testing for possible equivalence? rel_is_close(value1, value2, delta) # %delta close to each other.

I don't think they are quite the same thing.

rel_is_good(9, 10, .1) --> True rel_is_good(10, 9, .1) --> False

rel_is_close(9, 10, .1) --> True rel_is_close(10, 9, .1) --> True

In the "is close" case, it shouldn't matter what order the arguments are given. The delta is the distance from the larger number the smaller number is. (of the same sign)

So when calculating the relative error from two values, you want it to be consistent with the rel_is_close function.

rel_is_close(a, b, delta) <---> rel_err(a, b) <= delta

And you should not use the rel_err function in the rel_is_good function.

The next issue is, where does the numeric accuracy of the data, significant digits, and the languages accuracy (ULPs), come into the picture.

My intuition.. I need to test the idea to make a firmer claim.. is that in the case of is_good, you want to exclude the uncertain parts, but with is_close, you want to include the uncertain parts.

Two values "are close" if you can't tell one from the other with certainty. The is_close range includes any uncertainty.

A value is good if it's within a range with certainty. And this excludes any uncertainty.

This is where taking in consideration of an absolute delta comes in. The minimum range for both is the uncertainty of the data. But is_close and is_good do different things with it.

Of course all of this only applies if you agree with these definitions of is_close, and is_good. ;)

Cheers, Ron

_______________________________________________ Python-ideas mailing list Python...@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

--

Christopher Barker, Ph.D. Oceanographer

Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception

Chris....@noaa.gov

-- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov