Re: [Pythonideas] PEP 485: A Function for testing approximate equality
Hi folks,
Time for Take 2 (or is that take 200?)
No, I haven't dropped this ;) After the lengthy thread, I went through and tried to take into account all the thoughts and comments. The result is a PEP with a bunch more explanation, and some slightly different decisions.
TL;DR:
Please take a look and express your approval or disapproval (+1, +0, 0, 1, or whatever). Please keep in mind that this has been discussed a lot, so try to not to reiterate what's already been said. And I'd really like it if you only gave 1, thumbs down, etc, if you really think this is worse than not having anything at all, rather than you'd just prefer something a little different. Also, if you do think this version is worse than nothing, but a different choice or two could change that, then let us know what  maybe we can reach consensus on something slightly different.
Also keep in mid that the goal here (in the words of Nick Coghlan): """ the key requirement here should be "provide a binary float comparison function that is significantly less wrong than the current 'a == b'" """ No need to split hairs.
https://www.python.org/dev/peps/pep0485/
Full PEP below, and in gitHub (with example code, tests, etc.) here:
https://github.com/PythonCHB/close_pep
Full Detail: =========
Here are the big changes:
* Symmetric test. Essentially, whether you want the symmetric or asymmetric test depends on exactly what question you are asking, but I think we need to pick one. It makes little to no difference for most use cases (see below) , but I was persuaded by:
 Principle of least surprise  equality is symmetric, shouldn't "closeness" be too?  People don't want to have to remember what order to put the arguments even if it doesn't matter, you have to think about whether it matters. A symmetric test doesn't require that.  There are times when the order of comparison is not known  for example if the users wants to test a bunch of values all against eachother.
On the other hand  the asymmetric test is a better option only when you are specifically asking the question: is this (computed) value within a precise tolerance of another(known) value, and that tolerance is fairly large (i.e 1%  10%). While this was brought up on this thread, no one had an actual usecase like that. And is it really that hard to write:
known  computed <= 0.1* expected
NOTE: that my example code has a flag with which you can pick which test to use  I did that for experimentation, but I think we want a simple API here. As soon as you provide that option people need to concern themselves with what to use.
* Defaults: I set the default relative tolerance to 1e9  because that is a little more than half the precision available in a Python float, and it's the largest tolerance for which ALL the possible tests result in exactly the same result for all possible float values (see the PEP for the math). The default for absolute tolerance is 0.0. Better for people to get a failed test with a check for zero right away than accidentally use a totally inappropriate value.
Contentious Issues ===============
* The Symmetric vs. Asymmetric thing  some folks seemed to have string ideas about that, but i hope we can all agree that something is better than nothing, and symmetric is probably the least surprising. And take a look at the math in teh PEP  for small tolerance, which is the likely case for most tests  it literally makes no difference at all which test is used.
* Default for abs_tol  or even whether to have it or not. This is a tough one  there are a lot of usecases for people testing against zero, so it's a pity not to have it do something reasonable by default in that case. However, there really is no reasonable universal default.
As Guido said: """ For someone who for whatever reason is manipulating quantities that are in the range of 1e100, 1e12 is about as large as infinity. """ And testing against zero really requires a absolute tolerance test, which is not hard to simply write:
abs(my_val) <= some_tolerance
and is covered by the unittest assert already.
So why include an absolute_tolerance at all?  Because this is likely to be used inside a comprehension or in the a unittest sequence comparison, so it is very helpful to be able to test a bunch of values some of which may be zero, all with a single function with a single set of parameters. And a similar approach has proven to be useful for numpy and the statistics test module. And again, the default is abs_tol=0.0  if the user sticks with defaults, they won't get bitten.
* Also  not really contentious (I hope) but I left the details out about how the unittest assertion would work. That's because I don't really use unittest  I'm hoping someone else will write that. If it comes down to it, I can do it  it will probably look a lot like the existing sequence comparing assertions.
OK  I think that's it.
Let's see if we can drive this home.
Chris
PEP: 485 Title: A Function for testing approximate equality Version: $Revision$ LastModified: $Date$ Author: Christopher Barker Chris.Barker@noaa.gov Status: Draft Type: Standards Track ContentType: text/xrst Created: 20Jan2015 PythonVersion: 3.5 PostHistory:
Abstract ========
This PEP proposes the addition of a function to the standard library that determines whether one value is approximately equal or "close" to another value. It is also proposed that an assertion be added to the ``unittest.TestCase`` class to provide easy access for those using unittest for testing.
Rationale =========
Floating point values contain limited precision, which results in their being unable to exactly represent some values, and for error to accumulate with repeated computation. As a result, it is common advice to only use an equality comparison in very specific situations. Often a inequality comparison fits the bill, but there are times (often in testing) where the programmer wants to determine whether a computed value is "close" to an expected value, without requiring them to be exactly equal. This is common enough, particularly in testing, and not always obvious how to do it, so it would be useful addition to the standard library.
Existing Implementations 
The standard library includes the ``unittest.TestCase.assertAlmostEqual`` method, but it:
* Is buried in the unittest.TestCase class
* Is an assertion, so you can't use it as a general test at the command line, etc. (easily)
* Is an absolute difference test. Often the measure of difference requires, particularly for floating point numbers, a relative error, i.e "Are these two values within x% of eachother?", rather than an absolute error. Particularly when the magnatude of the values is unknown a priori.
The numpy package has the ``allclose()`` and ``isclose()`` functions, but they are only available with numpy.
The statistics package tests include an implementation, used for its unit tests.
One can also find discussion and sample implementations on Stack Overflow and other help sites.
Many other nonpython systems provide such a test, including the Boost C++ library and the APL language (reference?).
These existing implementations indicate that this is a common need and not trivial to write oneself, making it a candidate for the standard library.
Proposed Implementation =======================
NOTE: this PEP is the result of an extended discussion on the pythonideas list [1]_.
The new function will have the following signature::
is_close(a, b, rel_tolerance=1e9, abs_tolerance=0.0)
``a`` and ``b``: are the two values to be tested to relative closeness
``rel_tolerance``: is the relative tolerance  it is the amount of error allowed, relative to the magnitude a and b. For example, to set a tolerance of 5%, pass tol=0.05. The default tolerance is 1e8, which assures that the two values are the same within about 8 decimal digits.
``abs_tolerance``: is an minimum absolute tolerance level  useful for comparisons near zero.
Modulo error checking, etc, the function will return the result of::
abs(ab) <= max( rel_tolerance * min(abs(a), abs(b), abs_tolerance )
Handling of nonfinite numbers 
The IEEE 754 special values of NaN, inf, and inf will be handled according to IEEE rules. Specifically, NaN is not considered close to any other value, including NaN. inf and inf are only considered close to themselves.
Nonfloat types 
The primary usecase is expected to be floating point numbers. However, users may want to compare other numeric types similarly. In theory, it should work for any type that supports ``abs()``, comparisons, and subtraction. The code will be written and tested to accommodate these types:
* ``Decimal``: for Decimal, the tolerance must be set to a Decimal type.
* ``int``
* ``Fraction``
* ``complex``: for complex, ``abs(z)`` will be used for scaling and comparison.
Behavior near zero 
Relative comparison is problematic if either value is zero. By definition, no value is small relative to zero. And computationally, if either value is zero, the difference is the absolute value of the other value, and the computed absolute tolerance will be rel_tolerance times that value. reltolerance is always less than one, so the difference will never be less than the tolerance.
However, while mathematically correct, there are many use cases where a user will need to know if a computed value is "close" to zero. This calls for an absolute tolerance test. If the user needs to call this function inside a loop or comprehension, where some, but not all, of the expected values may be zero, it is important that both a relative tolerance and absolute tolerance can be tested for with a single function with a single set of parameters.
There is a similar issue if the two values to be compared straddle zero: if a is approximately equal to b, then a and b will never be computed as "close".
To handle this case, an optional parameter, ``abs_tolerance`` can be used to set a minimum tolerance used in the case of very small or zero computed absolute tolerance. That is, the values will be always be considered close if the difference between them is less than the abs_tolerance
The default absolute tolerance value is set to zero because there is no value that is appropriate for the general case. It is impossible to know an appropriate value without knowing the likely values expected for a given use case. If all the values tested are on order of one, then a value of about 1e8 might be appropriate, but that would be far too large if expected values are on order of 1e12 or smaller.
Any nonzero default might result in user's tests passing totally inappropriately. If, on the other hand a test against zero fails the first time with defaults, a user will be prompted to select an appropriate value for the problem at hand in order to get the test to pass.
NOTE: that the author of this PEP has resolved to go back over many of his tests that use the numpy ``all_close()`` function, which provides a default abs_tolerance, and make sure that the default value is appropriate.
If the user sets the rel_tolerance parameter to 0.0, then only the absolute tolerance will effect the result. While not the goal of the function, it does allow it to be used as a purely absolute tolerance check as well.
unittest assertion 
[need text here]
implementation 
A sample implementation is available (as of Jan 22, 2015) on gitHub:
https://github.com/PythonCHB/close_pep/blob/master
This implementation has a flag that lets the user select which relative tolerance test to apply  this PEP does not suggest that that be retained, but rather than the strong test be selected.
Relative Difference ===================
There are essentially two ways to think about how close two numbers are to eachother:
Absolute difference: simply ``abs(ab)``
Relative difference: ``abs(ab)/scale_factor`` [2]_.
The absolute difference is trivial enough that this proposal focuses on the relative difference.
Usually, the scale factor is some function of the values under consideration, for instance:
1) The absolute value of one of the input values
2) The maximum absolute value of the two
3) The minimum absolute value of the two.
4) The absolute value of the arithmetic mean of the two
These lead to the following possibilities for determining if two values, a and b, are close to each other.
1) ``abs(ab) <= tol*abs(a)``
2) ``abs(ab) <= tol * max( abs(a), abs(b) )``
3) ``abs(ab) <= tol * min( abs(a), abs(b) )``
4) ``abs(ab) <= tol * (a + b)/2``
NOTE: (2) and (3) can also be written as:
2) ``(abs(ab) <= tol*abs(a)) or (abs(ab) <= tol*abs(a))``
3) ``(abs(ab) <= tol*abs(a)) and (abs(ab) <= tol*abs(a))``
(Boost refers to these as the "weak" and "strong" formulations [3]_) These can be a tiny bit more computationally efficient, and thus are used in the example code.
Each of these formulations can lead to slightly different results. However, if the tolerance value is small, the differences are quite small. In fact, often less than available floating point precision.
How much difference does it make? 
When selecting a method to determine closeness, one might want to know how much of a difference it could make to use one test or the other  i.e. how many values are there (or what range of values) that will pass one test, but not the other.
The largest difference is between options (2) and (3) where the allowable absolute difference is scaled by either the larger or smaller of the values.
Define ``delta`` to be the difference between the allowable absolute tolerance defined by the larger value and that defined by the smaller value. That is, the amount that the two input values need to be different in order to get a different result from the two tests. ``tol`` is the relative tolerance value.
Assume that ``a`` is the larger value and that both ``a`` and ``b`` are positive, to make the analysis a bit easier. ``delta`` is therefore::
delta = tol * (ab)
or::
delta / tol = (ab)
The largest absolute difference that would pass the test: ``(ab)``, equals the tolerance times the larger value::
(ab) = tol * a
Substituting into the expression for delta::
delta / tol = tol * a
so::
delta = tol**2 * a
For example, for ``a = 10``, ``b = 9``, ``tol = 0.1`` (10%):
maximum tolerance ``tol * a == 0.1 * 10 == 1.0``
minimum tolerance ``tol * b == 0.1 * 9.0 == 0.9``
delta = ``(1.0  0.9) * 0.1 = 0.1`` or ``tol**2 * a = 0.1**2 * 10 = .01``
The absolute difference between the maximum and minimum tolerance tests in this case could be substantial. However, the primary use case for the proposed function is testing the results of computations. In that case a relative tolerance is likely to be selected of much smaller magnitude.
For example, a relative tolerance of ``1e8`` is about half the precision available in a python float. In that case, the difference between the two tests is ``1e8**2 * a`` or ``1e16 * a``, which is close to the limit of precision of a python float. If the relative tolerance is set to the proposed default of 1e9 (or smaller), the difference between the two tests will be lost to the limits of precision of floating point. That is, each of the four methods will yield exactly the same results for all values of a and b.
In addition, in common use, tolerances are defined to 1 significant figure  that is, 1e8 is specifying about 8 decimal digits of accuracy. So the difference between the various possible tests is well below the precision to which the tolerance is specified.
Symmetry 
A relative comparison can be either symmetric or nonsymmetric. For a symmetric algorithm:
``is_close_to(a,b)`` is always the same as ``is_close_to(b,a)``
If a relative closeness test uses only one of the values (such as (1) above), then the result is asymmetric, i.e. is_close_to(a,b) is not necessarily the same as is_close_to(b,a).
Which approach is most appropriate depends on what question is being asked. If the question is: "are these two numbers close to each other?", there is no obvious ordering, and a symmetric test is most appropriate.
However, if the question is: "Is the computed value within x% of this known value?", then it is appropriate to scale the tolerance to the known value, and an asymmetric test is most appropriate.
From the previous section, it is clear that either approach would
yield the same or similar results in the common use cases. In that case, the goal of this proposal is to provide a function that is least likely to produce surprising results.
The symmetric approach provide an appealing consistency  it mirrors the symmetry of equality, and is less likely to confuse people. A symmetric test also relieves the user of the need to think about the order in which to set the arguments. It was also pointed out that there may be some cases where the order of evaluation may not be well defined, for instance in the case of comparing a set of values all against each other.
There may be cases when a user does need to know that a value is within a particular range of a known value. In that case, it is easy enough to simply write the test directly::
if ab <= tol*a:
(assuming a > b in this case). There is little need to provide a function for this particular case.
This proposal uses a symmetric test.
Which symmetric test? 
There are three symmetric tests considered:
The case that uses the arithmetic mean of the two values requires that the value be either added together before dividing by 2, which could result in extra overflow to inf for very large numbers, or require each value to be divided by two before being added together, which could result in underflow to inf for very small numbers. This effect would only occur at the very limit of float values, but it was decided there as no benefit to the method worth reducing the range of functionality.
This leaves the boost "weak" test (2) or using the smaller value to scale the tolerance, or the Boost "strong" (3) test, which uses the smaller of the values to scale the tolerance. For small tolerance, they yield the same result, but this proposal uses the boost "strong" test case: it is symmetric and provides a slightly stricter criteria for tolerance.
Defaults ========
Default values are required for the relative and absolute tolerance.
Relative Tolerance Default 
The relative tolerance required for two values to be considered "close" is entirely usecase dependent. Nevertheless, the relative tolerance needs to be less than 1.0, and greater than 1e16 (approximate precision of a python float). The value of 1e9 was selected because it is the largest relative tolerance for which the various possible methods will yield the same result, and it is also about half of the precision available to a python float. In the general case, a good numerical algorithm is not expected to lose more than about half of available digits of accuracy, and if a much larger tolerance is acceptable, the user should be considering the proper value in that case. Thus 1e9 is expected to "just work" for many cases.
Absolute tolerance default 
The absolute tolerance value will be used primarily for comparing to zero. The absolute tolerance required to determine if a value is "close" to zero is entirely usecase dependent. There is also essentially no bounds to the useful range  expected values would conceivably be anywhere within the limits of a python float. Thus a default of 0.0 is selected.
If, for a given use case, a user needs to compare to zero, the test will be guaranteed to fail the first time, and the user can select an appropriate value.
It was suggested that comparing to zero is, in fact, a common use case (evidence suggest that the numpy functions are often used with zero). In this case, it would be desirable to have a "useful" default. Values around 1e8 were suggested, being about half of floating point precision for values of around value 1.
However, to quote The Zen: "In the face of ambiguity, refuse the temptation to guess." Guessing that users will most often be concerned with values close to 1.0 would lead to spurious passing tests when used with smaller values  this is potentially more damaging than requiring the user to thoughtfully select an appropriate value.
Expected Uses =============
The primary expected use case is various forms of testing  "are the results computed near what I expect as a result?" This sort of test may or may not be part of a formal unit testing suite. Such testing could be used oneoff at the command line, in an iPython notebook, part of doctests, or simple assets in an ``if __name__ == "__main__"`` block.
The proposed unitest.TestCase assertion would have course be used in unit testing.
It would also be an appropriate function to use for the termination criteria for a simple iterative solution to an implicit function::
guess = something while True: new_guess = implicit_function(guess, *args) if is_close(new_guess, guess): break guess = new_guess
Inappropriate uses 
One use case for floating point comparison is testing the accuracy of a numerical algorithm. However, in this case, the numerical analyst ideally would be doing careful error propagation analysis, and should understand exactly what to test for. It is also likely that ULP (Unit in the Last Place) comparison may be called for. While this function may prove useful in such situations, It is not intended to be used in that way.
Other Approaches ================
``unittest.TestCase.assertAlmostEqual`` 
( https://docs.python.org/3/library/unittest.html#unittest.TestCase.assertAlmo... )
Tests that values are approximately (or not approximately) equal by computing the difference, rounding to the given number of decimal places (default 7), and comparing to zero.
This method is purely an absolute tolerance test, and does not address the need for a relative tolerance test.
numpy ``is_close()`` 
http://docs.scipy.org/doc/numpydev/reference/generated/numpy.isclose.html
The numpy package provides the vectorized functions is_close() and all_close, for similar use cases as this proposal:
``isclose(a, b, rtol=1e05, atol=1e08, equal_nan=False)``
Returns a boolean array where two arrays are elementwise equal within a tolerance.
The tolerance values are positive, typically very small numbers. The relative difference (rtol * abs(b)) and the absolute difference atol are added together to compare against the absolute difference between a and b
In this approach, the absolute and relative tolerance are added together, rather than the ``or`` method used in this proposal. This is computationally more simple, and if relative tolerance is larger than the absolute tolerance, then the addition will have no effect. However, if the absolute and relative tolerances are of similar magnitude, then the allowed difference will be about twice as large as expected.
This makes the function harder to understand, with no computational advantage in this context.
Even more critically, if the values passed in are small compared to the absolute tolerance, then the relative tolerance will be completely swamped, perhaps unexpectedly.
This is why, in this proposal, the absolute tolerance defaults to zero  the user will be required to choose a value appropriate for the values at hand.
Boost floatingpoint comparison 
The Boost project ( [3]_ ) provides a floating point comparison function. Is is a symmetric approach, with both "weak" (larger of the two relative errors) and "strong" (smaller of the two relative errors) options. This proposal uses the Boost "strong" approach. There is no need to complicate the API by providing the option to select different methods when the results will be similar in most cases, and the user is unlikely to know which to select in any case.
Alternate Proposals 
A Recipe '''''''''
The primary alternate proposal was to not provide a standard library function at all, but rather, provide a recipe for users to refer to. This would have the advantage that the recipe could provide and explain the various options, and let the user select that which is most appropriate. However, that would require anyone needing such a test to, at the very least, copy the function into their code base, and select the comparison method to use.
In addition, adding the function to the standard library allows it to be used in the ``unittest.TestCase.assertIsClose()`` method, providing a substantial convenience to those using unittest.
``zero_tol`` ''''''''''''
One possibility was to provide a zero tolerance parameter, rather than the absolute tolerance parameter. This would be an absolute tolerance that would only be applied in the case of one of the arguments being exactly zero. This would have the advantage of retaining the full relative tolerance behavior for all nonzero values, while allowing tests against zero to work. However, it would also result in the potentially surprising result that a small value could be "close" to zero, but not "close" to an even smaller value. e.g., 1e10 is "close" to zero, but not "close" to 1e11.
No absolute tolerance '''''''''''''''''''''
Given the issues with comparing to zero, another possibility would have been to only provide a relative tolerance, and let every comparison to zero fail. In this case, the user would need to do a simple absolute test: `abs(val) < zero_tol` in the case where the comparison involved zero.
However, this would not allow the same call to be used for a sequence of values, such as in a loop or comprehension, or in the ``TestCase.assertClose()`` method. Making the function far less useful. It is noted that the default abs_tolerance=0.0 achieves the same effect if the default is not overidden.
Other tests ''''''''''''
The other tests considered are all discussed in the Relative Error section above.
References ==========
.. [1] Pythonideas list discussion threads
https://mail.python.org/pipermail/pythonideas/2015January/030947.html
https://mail.python.org/pipermail/pythonideas/2015January/031124.html
https://mail.python.org/pipermail/pythonideas/2015January/031313.html
.. [2] Wikipedia page on relative difference
http://en.wikipedia.org/wiki/Relative_change_and_difference
.. [3] Boost project floatingpoint comparison algorithms
http://www.boost.org/doc/libs/1_35_0/libs/test/doc/components/test_tools/flo...
.. Bruce Dawson's discussion of floating point.
https://randomascii.wordpress.com/2012/02/25/comparingfloatingpointnumber...
Copyright =========
This document has been placed in the public domain.
.. Local Variables: mode: indentedtext indenttabsmode: nil sentenceenddoublespace: t fillcolumn: 70 coding: utf8
I think this looks good and I hope it'll get much approval. Thanks for hanging in there! I did a quick review:
 Which module should this go into? (I think math, but I don't think you state it.)
 Sometimes the default relative tolerance is described as 1e8, sometimes as 1e9.
 The formula with "abs(ab) <= max( ... min(..." is missing an allimportant close parenthesis. I presume it goes before the last comma. :)
 What if a tolerance is inf or nan? What if all args are inf or nan?
 I'd argue for allowing the tolerances to be floats even if a and b are Decimals. There's not much difference in the outcome. Also, what if exactly one of a and b is Decimal?
 Your language about complex confuses me. What is z? I'd expect the following behavior:  tolerances cannot be complex  abs(ab) is the right thing to use Or maybe you meant to say that abs() of the tolerances will be used if they are complex? That makes some sense in case someone just mindlessly passes a real cast to complex (i.e. with a zero imaginary part, or nearzero in case it's the outcome of a very fuzzy computation). But still. Maybe you can just give the formula to be used for complex args, if it is in fact any different from the one for reals?
 For unittests are you proposing to replace assertAlmostEquals or deprecate that and add a new assertIsCloseTo? (I hope the latter.)
 "there as no benefit"  as>was
I've updated the copy on gitHub with these notes.
 Which module should this go into? (I think math, but I don't think you
state it.)
added to text.
 Sometimes the default relative tolerance is described as 1e8, sometimes
as 1e9.
that's what I get for changing it  fixed.
 The formula with "abs(ab) <= max( ... min(..." is missing an
allimportant close parenthesis. I presume it goes before the last comma. :)
fixed.
 What if a tolerance is inf or nan? What if all args are inf or nan?
0.0 < rel_tol < 1.0)
I've enforced this in the sample code  it's not strictly necessary, but what does a negative relative tolerance mean, anyway, and a tolerance over 1.0 has some odd behaviour with zero and could hardly be called "close" anyway. Text added to make this clear.
if a and/or b are inf or nan, it follows IEE754 (which mostly "just works" with no special casing in the code.
 I'd argue for allowing the tolerances to be floats even if a and b are
Decimals. There's not much difference in the outcome. Also, what if exactly one of a and b is Decimal?
This resulted from tests failing with trying to convert a float to a Decimal in the code. Though I can't find that problem right now. But maybe type dispatch is a better way to handle that.
 Your language about complex confuses me. What is z?
"standard" notation for an arbitrary complex value  I've clarified the language.
I'd expect the following behavior:
 tolerances cannot be complex
 abs(ab) is the right thing to use
Or maybe you meant to say that abs() of the tolerances will be used if they are complex? That makes some sense in case someone just mindlessly passes a real cast to complex (i.e. with a zero imaginary part, or nearzero in case it's the outcome of a very fuzzy computation). But still. Maybe you can just give the formula to be used for complex args, if it is in fact any different from the one for reals?
it's the same, with the exception of using isinf from cmath. At this point, my example code multiplies the tolerance times the tested values, then takes the absolute values  so a complex tolerance will do complex multiplication, then use the magnitude from that  I'll have to think about whether that is reasonable.
 For unittests are you proposing to replace assertAlmostEquals or
deprecate that and add a new assertIsCloseTo? (I hope the latter.
Add an assertIsCloseTo  whether assertAlmostEquals should be deprecated I have no opinion on. Though it is useful, as it acts over a sequence  I"m guessing fols will want to keep it. Then we'll need them to reference each other in the docs.
On Thu, Feb 5, 2015 at 10:46 AM, Nikolaus Rath Nikolaus@rath.org wrote:
 ``(abs(ab) <= tol*abs(a)) or (abs(ab) <= tol*abs(a))``
I think the last 'a' should be a 'b'?
indeed  fixed.
Thanks,
Chris
On Thursday, February 5, 2015 11:47 AM, Chris Barker chris.barker@noaa.gov wrote:
 For unittests are you proposing to replace assertAlmostEquals or deprecate that and add a new assertIsCloseTo? (I hope the latter.
Add an assertIsCloseTo  whether assertAlmostEquals should be deprecated I have no opinion on. Though it is useful, as it acts over a sequence  I"m guessing fols will want to keep it.
Why not have assertIsCloseTo act over a sequence in exactly the same way as assertAlmostEquals, in which case there's no reason to keep the latter (except for existing unit tests that already use it, but presumably a long period of deprecation is sufficient there)?
On Thu, Feb 5, 2015 at 2:49 PM, Andrew Barnert abarnert@yahoo.com wrote:
 For unittests are you proposing to replace assertAlmostEquals or
deprecate that and add a new assertIsCloseTo? (I hope the latter.
Add an assertIsCloseTo  whether assertAlmostEquals should be deprecated
I have no opinion on. Though it is useful, as it acts over a sequence  I"m guessing fols will want to keep it.
Why not have assertIsCloseTo act over a sequence in exactly the same way as assertAlmostEquals, in which case there's no reason to keep the latter (except for existing unit tests that already use it, but presumably a long period of deprecation is sufficient there)?
Because assertAlmostEquals() is an absolute tolerance test  not a relative one  and it lets (encourages?) users to specify number of decimal places (or optionally install a explicit absolute tolerance). I assume this is all there because someone found it useful.
Again, I have little opinion about all this  I don't like/use unitest anyway. I"m hoping someone else will take up the mantle of what to do with unittest...
Chris
On Thu, Feb 5, 2015 at 4:11 PM, Chris Barker chris.barker@noaa.gov wrote:
On Thu, Feb 5, 2015 at 2:49 PM, Andrew Barnert abarnert@yahoo.com wrote:
 For unittests are you proposing to replace assertAlmostEquals or
deprecate that and add a new assertIsCloseTo? (I hope the latter.
Add an assertIsCloseTo  whether assertAlmostEquals should be
deprecated I have no opinion on. Though it is useful, as it acts over a sequence  I"m guessing fols will want to keep it.
Why not have assertIsCloseTo act over a sequence in exactly the same way as assertAlmostEquals, in which case there's no reason to keep the latter (except for existing unit tests that already use it, but presumably a long period of deprecation is sufficient there)?
Because assertAlmostEquals() is an absolute tolerance test  not a relative one  and it lets (encourages?) users to specify number of decimal places (or optionally install a explicit absolute tolerance). I assume this is all there because someone found it useful.
Again, I have little opinion about all this  I don't like/use unitest anyway. I"m hoping someone else will take up the mantle of what to do with unittest...
OK, let's just drop the idea of adding anything to unittest/case.py from the PEP. People can just write self.assertTrue(math.isclose(...)). Or if they're using py.test they can write assert math.isclose(...).
On Thu, Feb 5, 2015 at 11:39 AM, Chris Barker chris.barker@noaa.gov wrote:
Add an assertIsCloseTo  whether assertAlmostEquals should be deprecated I have no opinion on. Though it is useful, as it acts over a sequence [...]
Are you sure? I don't see any special handling of sequences in its docs or code...
n
On Thu, Feb 5, 2015 at 2:59 PM, Nathaniel Smith njs@pobox.com wrote:
On Thu, Feb 5, 2015 at 11:39 AM, Chris Barker chris.barker@noaa.gov wrote:
Add an assertIsCloseTo  whether assertAlmostEquals should be
deprecated I
have no opinion on. Though it is useful, as it acts over a sequence [...]
Are you sure? I don't see any special handling of sequences in its docs or code...
The one in unittest doesn't have this feature. However there's assertApproxEqual defined locally in the test/test_statistics.py package that does. Since the other is only meant for local use in test_statistics.py, we should leave it alone. The one in unittest is pretty bad but widely used; deprecating it won't have any teeth.
On Thu, Feb 5, 2015 at 3:05 PM, Guido van Rossum guido@python.org wrote:
Are you sure? I don't see any special handling of sequences in its
docs or code...
The one in unittest doesn't have this feature. However there's assertApproxEqual defined locally in the test/test_statistics.py package that does.
Indeed  it looks like I confused those two  sorry about that.
Chris
Since the other is only meant for local use in test_statistics.py, we should leave it alone. The one in unittest is pretty bad but widely used; deprecating it won't have any teeth.
 Guido van Rossum (python.org/~guido)
On Thu, Feb 05, 2015 at 11:39:43AM 0800, Chris Barker wrote:
 What if a tolerance is inf or nan? What if all args are inf or nan?
0.0 < rel_tol < 1.0)
I can just about see the point in restricting rel_tol to the closed interval 0...1, but not the open interval. Mathematically, setting the tolerance to 0 should just degrade gracefully to exact equality, and a tolerance of 1 is nothing special at all.
Values larger than 1 aren't often useful, but there really is no reason to exclude tolerances larger than 1. "Give or take 300%" (ie. rel_tol=3.0) is a pretty big tolerance, but it is welldefined: a difference of 299% is "close enough", 301% is "too far".
You might argue that if you want exact equality, you shouldn't use a tolerance at all but just use == instead. But consider a case where you might be trying calculations repeatedly and decreasing the tolerance until it fails:
# doesn't handle termination properly rel_tol = pick_some_number while close_to(something, something_else, rel_tol): rel_tol /= 2
In some cases, your calculation may actually be exact, and it simplies the code if close_to(a, b, rel_tol) degrades nicely to handle the exact case rather than you needing to switch between close_to and == by hand.
Negative error tolerances, on the other hand, do seem to be meaningless and should be prevented.
I've enforced this in the sample code  it's not strictly necessary, but what does a negative relative tolerance mean, anyway, and a tolerance over 1.0 has some odd behaviour with zero and could hardly be called "close" anyway. Text added to make this clear.
I don't see why a tolerance over 1 should behave any more oddly with zero than a tolerance under 1.
And as for "close", some people would argue that rel_tol=0.5 is "hardly close". Sometimes you might accept anything within an order of magnitude the expected value: anything up to 10*x is "close enough". Or 20 times. Who knows?
(E.g. "guess the number of grains of sand on this beach".) Any upper limit you put in is completely arbitrary, and this function shouldn't be in the business of telling people how much error they are allowed to tolerate. It's easy for people to put in their own restrictions, but hard for them to work around yours.
if a and/or b are inf or nan, it follows IEE754 (which mostly "just works" with no special casing in the code.
That is very nice.
On Thu, Feb 5, 2015 at 4:44 PM, Steven D'Aprano steve@pearwood.info wrote:
0.0 < rel_tol < 1.0)
I can just about see the point in restricting rel_tol to the closed interval 0...1, but not the open interval. Mathematically, setting the tolerance to 0 should just degrade gracefully to exact equality,
sure  no harm done there.
and a tolerance of
1 is nothing special at all.
well, I ended up putting that in because it turns out with the "weak" test, then anything compares as "close" to zero:
tol>=1.0 a = anything b = 0.0
min( abs(a), abs(b) ) = 0.0
abs(ab) = a
tol * a >= a
abs(ab) <= tol * a
Granted, that's actually the best argument yet for using the strong test  which I am suggesting, though I haven't thought out what that will do in the case of large tolerances.
Values larger than 1 aren't often useful, but there really is no reason to exclude tolerances larger than 1. "Give or take 300%" (ie. rel_tol=3.0) is a pretty big tolerance, but it is welldefined: a difference of 299% is "close enough", 301% is "too far".
yes it is, but then the whole weak vs string vs asymmetric test becomes important. From my math the "delta" between the weak and strong tests goes with tolerance**2 * max(a,b). So if the tolerance is >=1, then it makes a big difference which test you choose. IN fact:
Is a within 300% of b makes sense, but "are a and b within 300% of eachother" is poorly defined.
The goal of this is to provide an easy way for users to test if two values are close  not to provide a general purpose relative difference function. and keeping the scope small keeps things cleaner.
You might argue that if you want exact equality, you shouldn't use a
tolerance at all but just use == instead. But consider a case where you might be trying calculations repeatedly and decreasing the tolerance until it fails:
sure  then might as well include 0.0
Negative error tolerances, on the other hand, do seem to be meaningless and should be prevented.
you could just take the abs(rel_tol), but really? what's the point?
I don't see why a tolerance over 1 should behave any more oddly with zero than a tolerance under 1.
see above.
And as for "close", some people would argue that rel_tol=0.5 is "hardly close". Sometimes you might accept anything within an order of magnitude the expected value: anything up to 10*x is "close enough". Or 20 times. Who knows?
with some more thought, we may be able to let rel_tol be anything > 0.0 with the string test. I"ll have to investigate more if folks think this is important.
(E.g. "guess the number of grains of sand on this beach".) Any upper limit you put in is completely arbitrary,
somehow one doesn't feel arbitrary to me  numbers aren't close if the difference between them is larger than the largest of the numbers  not arbitrary, maybe unneccesary , but not arbirtrary
and this function shouldn't be in the business of telling people how much error they are allowed to tolerate. It's easy for people to put in their own restrictions, but hard for them to work around yours.
if a and/or b are inf or nan, it follows IEE754 (which mostly "just
works"
with no special casing in the code.
That is very nice.
 Steve _______________________________________________ Pythonideas mailing list Pythonideas@python.org https://mail.python.org/mailman/listinfo/pythonideas Code of Conduct: http://python.org/psf/codeofconduct/
On Thu, Feb 05, 2015 at 05:12:32PM 0800, Chris Barker wrote:
On Thu, Feb 5, 2015 at 4:44 PM, Steven D'Aprano steve@pearwood.info wrote:
0.0 < rel_tol < 1.0)
I can just about see the point in restricting rel_tol to the closed interval 0...1, but not the open interval. Mathematically, setting the tolerance to 0 should just degrade gracefully to exact equality,
sure  no harm done there.
and a tolerance of 1 is nothing special at all.
well, I ended up putting that in because it turns out with the "weak" test, then anything compares as "close" to zero:
Okay. Maybe that means that the "weak test" is not useful if one of the numbers is zero. Neither is a relative tolerance, or an ULP calculation.
tol>=1.0 a = anything b = 0.0 min( abs(a), abs(b) ) = 0.0 abs(ab) = a
That is incorrect. abs(ab) for b == 0 is abs(a), not a.
tol * a >= a abs(ab) <= tol * a
If and only if a >= 0. Makes perfect mathematical sense, even if it's not useful. That's an argument for doing what Bruce Dawson says, and comparing against the maximum of a and b, not the minimum.
Granted, that's actually the best argument yet for using the strong test  which I am suggesting, though I haven't thought out what that will do in the case of large tolerances.
It should work exactly the same as for small tolerances, except larger *wink*
Values larger than 1 aren't often useful, but there really is no reason to exclude tolerances larger than 1. "Give or take 300%" (ie. rel_tol=3.0) is a pretty big tolerance, but it is welldefined: a difference of 299% is "close enough", 301% is "too far".
yes it is, but then the whole weak vs string vs asymmetric test becomes important.
Um, yes? I know Guido keeps saying that the difference is unimportant, but I think he is wrong: at the edges, the way you determine "close to" makes a difference whether a and b are considered close or not. If you care enough to specify a specific tolerance (say, 2.3e4), as opposed to plucking a round number out of thin air, then you care about the edge cases. I'm not entirely sure what to do about it, but my sense is that we should do something.
From my math the "delta" between the weak and strong tests goes with tolerance**2 * max(a,b). So if the tolerance is >=1, then it makes a big difference which test you choose. IN fact:
Is a within 300% of b makes sense, but "are a and b within 300% of eachother" is poorly defined.
No more so that "a and b within 1% of each other". It's just a shorthand. What I mean by "of each other" is the method recommended by Bruce Dawson, use the larger of a and b, what Boost(?) and you are calling the "strong test".
[...]
Negative error tolerances, on the other hand, do seem to be meaningless and should be prevented.
you could just take the abs(rel_tol), but really? what's the point?
No no, I agree with you that tolerances (relative or absolute) should prohibit negative values. Or complex ones for that matter.
(E.g. "guess the number of grains of sand on this beach".) Any upper limit you put in is completely arbitrary,
somehow one doesn't feel arbitrary to me  numbers aren't close if the difference between them is larger than the largest of the numbers  not arbitrary, maybe unneccesary , but not arbirtrary
Consider one way of detecting outliers in numeric data: any number more than X standard deviations from the mean in either direction may be an outlier.
py> import statistics py> data = [1, 2, 100, 100, 100, 101, 102, 103, 104, 500, 100000] py> m = statistics.mean(data) py> tol = 3*statistics.stdev(data) py> [x for x in data if abs(xm) > tol] [100000] py> m, tol, tol/m (9201.181818181818, 90344.55455462009, 9.818798969508077)
tol/m is, of course, the error tolerance relative to m, which for the sake of the argument we are treating as the "known value": anything more than 9.818... times the mean is probably an outlier.
Now, the above uses an absolute tolerance, but I should be able to get the same results from a relative tolerance of 9.818... depending on which is more convenient to work with at the time.
Steven, I can't take it any longer. It is just absolutely ridiculous how much discussion we've already seen about a function that's a single line of code. I'll give you three choices. You can vote +1, 0 or 1. No more discussion. If you still keep picking on details I'll just kill the PEP to be done with the discussion.
On Thu, Feb 5, 2015 at 8:48 PM, Steven D'Aprano steve@pearwood.info wrote:
On Thu, Feb 05, 2015 at 05:12:32PM 0800, Chris Barker wrote:
On Thu, Feb 5, 2015 at 4:44 PM, Steven D'Aprano steve@pearwood.info
wrote:
0.0 < rel_tol < 1.0)
I can just about see the point in restricting rel_tol to the closed interval 0...1, but not the open interval. Mathematically, setting the tolerance to 0 should just degrade gracefully to exact equality,
sure  no harm done there.
and a tolerance of 1 is nothing special at all.
well, I ended up putting that in because it turns out with the "weak"
test,
then anything compares as "close" to zero:
Okay. Maybe that means that the "weak test" is not useful if one of the numbers is zero. Neither is a relative tolerance, or an ULP calculation.
tol>=1.0 a = anything b = 0.0 min( abs(a), abs(b) ) = 0.0 abs(ab) = a
That is incorrect. abs(ab) for b == 0 is abs(a), not a.
tol * a >= a abs(ab) <= tol * a
If and only if a >= 0. Makes perfect mathematical sense, even if it's not useful. That's an argument for doing what Bruce Dawson says, and comparing against the maximum of a and b, not the minimum.
Granted, that's actually the best argument yet for using the strong test

which I am suggesting, though I haven't thought out what that will do in the case of large tolerances.
It should work exactly the same as for small tolerances, except larger *wink*
Values larger than 1 aren't often useful, but there really is no reason to exclude tolerances larger than 1. "Give or take 300%" (ie. rel_tol=3.0) is a pretty big tolerance, but it is welldefined: a difference of 299% is "close enough", 301% is "too far".
yes it is, but then the whole weak vs string vs asymmetric test becomes important.
Um, yes? I know Guido keeps saying that the difference is unimportant, but I think he is wrong: at the edges, the way you determine "close to" makes a difference whether a and b are considered close or not. If you care enough to specify a specific tolerance (say, 2.3e4), as opposed to plucking a round number out of thin air, then you care about the edge cases. I'm not entirely sure what to do about it, but my sense is that we should do something.
From my math the "delta" between the weak and strong tests goes with tolerance**2 * max(a,b). So if the tolerance is >=1, then it
makes a
big difference which test you choose. IN fact:
Is a within 300% of b makes sense, but "are a and b within 300% of eachother" is poorly defined.
No more so that "a and b within 1% of each other". It's just a shorthand. What I mean by "of each other" is the method recommended by Bruce Dawson, use the larger of a and b, what Boost(?) and you are calling the "strong test".
[...]
Negative error tolerances, on the other hand, do seem to be meaningless and should be prevented.
you could just take the abs(rel_tol), but really? what's the point?
No no, I agree with you that tolerances (relative or absolute) should prohibit negative values. Or complex ones for that matter.
(E.g. "guess the number of grains of sand on this beach".) Any upper limit you put in is completely arbitrary,
somehow one doesn't feel arbitrary to me  numbers aren't close if the difference between them is larger than the largest of the numbers  not arbitrary, maybe unneccesary , but not arbirtrary
Consider one way of detecting outliers in numeric data: any number more than X standard deviations from the mean in either direction may be an outlier.
py> import statistics py> data = [1, 2, 100, 100, 100, 101, 102, 103, 104, 500, 100000] py> m = statistics.mean(data) py> tol = 3*statistics.stdev(data) py> [x for x in data if abs(xm) > tol] [100000] py> m, tol, tol/m (9201.181818181818, 90344.55455462009, 9.818798969508077)
tol/m is, of course, the error tolerance relative to m, which for the sake of the argument we are treating as the "known value": anything more than 9.818... times the mean is probably an outlier.
Now, the above uses an absolute tolerance, but I should be able to get the same results from a relative tolerance of 9.818... depending on which is more convenient to work with at the time.
 Steve _______________________________________________ Pythonideas mailing list Pythonideas@python.org https://mail.python.org/mailman/listinfo/pythonideas Code of Conduct: http://python.org/psf/codeofconduct/
On 02/05/2015 06:44 PM, Steven D'Aprano wrote:
On Thu, Feb 05, 2015 at 11:39:43AM 0800, Chris Barker wrote:
 What if a tolerance is inf or nan? What if all args are inf or nan?
0.0 < rel_tol < 1.0)
I can just about see the point in restricting rel_tol to the closed interval 0...1, but not the open interval. Mathematically, setting the tolerance to 0 should just degrade gracefully to exact equality, and a tolerance of 1 is nothing special at all.
I Agree.
Values larger than 1 aren't often useful, but there really is no reason to exclude tolerances larger than 1. "Give or take 300%" (ie. rel_tol=3.0) is a pretty big tolerance, but it is welldefined: a difference of 299% is "close enough", 301% is "too far".
I agree here too.
The question I have is, is it better to get an excption, and possibly need to wrap the isclose() in a tryexcept, or better to let it go above 1?
If this is used in many places, I can see wrapping it in tryexcepts as a bigger problem. The same goes for when both tolerances are zero.
You might argue that if you want exact equality, you shouldn't use a tolerance at all but just use == instead. But consider a case where you might be trying calculations repeatedly and decreasing the tolerance until it fails:
# doesn't handle termination properly rel_tol = pick_some_number while close_to(something, something_else, rel_tol): rel_tol /= 2
In some cases, your calculation may actually be exact, and it simplies the code if close_to(a, b, rel_tol) degrades nicely to handle the exact case rather than you needing to switch between close_to and == by hand.
Yes.. +1
Negative error tolerances, on the other hand, do seem to be meaningless and should be prevented.
No they are well behaved. It's just a measurement from the other end of the board. ;)
t = .1 (10  10 * .1, 10 + 10 * .1) (9, 11)
t = .1 (10  10 * .1, 10 + 10 * .1) (11, 9)
So you get the same range either way. It works out in the isclose calculations just fine, a negative tolerance would give the same result as the positive of the same value.
Again, is this a place where we would want an exception, and would we need to write it like isclose(a, b, rel_tolerance=abs(t)) any place where the tolerance value might go negative, or would we like it to be more forgiving and easier to use?
I've enforced this in the sample code  it's not strictly necessary, but what does a negative relative tolerance mean, anyway, and a tolerance over 1.0 has some odd behaviour with zero and could hardly be called "close" anyway. Text added to make this clear.
I don't see why a tolerance over 1 should behave any more oddly with zero than a tolerance under 1.
And as for "close", some people would argue that rel_tol=0.5 is "hardly close". Sometimes you might accept anything within an order of magnitude the expected value: anything up to 10*x is "close enough". Or 20 times. Who knows?
(E.g. "guess the number of grains of sand on this beach".) Any upper limit you put in is completely arbitrary, and this function shouldn't be in the business of telling people how much error they are allowed to tolerate. It's easy for people to put in their own restrictions, but hard for them to work around yours.
I agree with this idea in principle. I think weather we want to handle exceptions or not is more of an issue though.
If it does allow higher tolerances, then it does need to handle them correctly. I don't see that as a difficult problem though.
I prefer the weak version myself. Because if you graph the result of True values for rel_tolerance=1. You get a graph where all like signed numbers are close. A tolerance of .5 gives a graph of the fifty percent of middle like signed numbers. And you can think of it as a percentage of the larger value. Which tends to be easier than thinking about percent increase.
Also this recommends using this method.
http://cfaq.com/fp/fpequal.html
So for the weak method and increasing values of tolerance:
A graphs of all True values fill from the like signed diagonal and converges on the unlike signed diagonal when rel_tolerance=2. Values above that don't change anything, they are all still true. So rel_tolerance=1 gives True for all like signed numbers. And all unlike signed number is the graph rel_tolerance=2 minus the graph of rel_tolerance=1.
For the strong method, and increasing values of tolerance:
The graphs fill from the like signed diagonal axis first until t=2, (the like signed quadrants are not filled yet), then it also starts filling from the unlike signed axis. So t=3 shows an X pattern, with a wider area in the like signed quadrants. I'm not sure what the upper value of t is needed so all values are True? It may be inf,
Keep in mind these graphs are the sum of comparing all the different values given a particular value of t.
So it there isn't a reason to limit the range of the tolerance, it will work just fine if we don't.
It's much harder to pick a strong tolerance that gives some sort of specific behaviour. For that reason I prefer the weak version much more than the strong version.
Cheers, Ron
On Thu, Feb 05, 2015 at 09:53:49PM 0600, Ron Adam wrote:
I prefer the weak version myself. Because if you graph the result of True values for rel_tolerance=1. You get a graph where all like signed numbers are close. A tolerance of .5 gives a graph of the fifty percent of middle like signed numbers. And you can think of it as a percentage of the larger value. Which tends to be easier than thinking about percent increase.
I'm afraid I don't understand this description.
Also this recommends using this method.
http://cfaq.com/fp/fpequal.html
You stopped reading too soon:
[quote] Doug Gwyn suggests using the following ``relative difference'' function. It returns the relative difference of two real numbers: 0.0 if they are exactly the same, otherwise the ratio of the difference to the larger of the two. [end quote]
Anyone got a copy of Knuth handy and check the reference given above?
Knuth Sec. 4.2.2 pp. 2178
I'd like to know what he has to say.
On 05/02/2015 18:56, Guido van Rossum wrote:
 Which module should this go into? (I think math, but I don't think
you state it.)
In that case it should be isclose(), for consistency with the other math.is* functions. But wherever we put it, I would suggest to change to isclose() because in the standard library we usually have issomething() instead of is_something().
On Thu, Feb 5, 2015 at 12:14 PM, Marco Buttu marco.buttu@gmail.com wrote:
On 05/02/2015 18:56, Guido van Rossum wrote:
 Which module should this go into? (I think math, but I don't think you
state it.)
In that case it should be isclose(), for consistency with the other math.is* functions. But wherever we put it, I would suggest to change to isclose() because in the standard library we usually have issomething() instead of is_something().
I was going with PEP8: "Function names should be lowercase, with words separated by underscores as necessary to improve readability." But consistency with other functions in the lib (or specifically in the same module) is probably better, yes.
Chris
 Marco Buttu
INAFOsservatorio Astronomico di Cagliari Via della Scienza n. 5, 09047 Selargius (CA) Phone: 070 711 80 217 Email: mbuttu@oacagliari.inaf.it
Pythonideas mailing list Pythonideas@python.org https://mail.python.org/mailman/listinfo/pythonideas Code of Conduct: http://python.org/psf/codeofconduct/
On 02/05/2015 01:03 PM, Chris Barker wrote:
On Thu, Feb 5, 2015 at 12:14 PM, Marco Buttu <marco.buttu@gmail.com mailto:marco.buttu@gmail.com> wrote:
On 05/02/2015 18:56, Guido van Rossum wrote:  Which module should this go into? (I think math, but I don't think you state it.) In that case it should be isclose(), for consistency with the other math.is <http://math.is>* functions. But wherever we put it, I would suggest to change to isclose() because in the standard library we usually have issomething() instead of is_something().
I was going with PEP8: "Function names should be lowercase, with words separated by underscores as necessary to improve readability." But consistency with other functions in the lib (or specifically in the same module) is probably better, yes.
PEP8 on consistency [1] (second paragraph):
A style guide is about consistency. Consistency with this style guide is important. Consistency within a project is more important. Consistency within one module or function is most important.
[1] https://www.python.org/dev/peps/pep0008/#id10
 ~Ethan~
On Thu, Feb 5, 2015 at 1:33 PM, Ethan Furman ethan@stoneleaf.us wrote:
I was going with PEP8: "Function names should be lowercase, with words
separated by underscores as necessary to improve
readability." But consistency with other functions in the lib (or
specifically in the same module) is probably better, yes.
PEP8 on consistency [1] (second paragraph):
A style guide is about consistency. Consistency with this style guide is important. Consistency within a project is more important. Consistency within one module or function is most important.
Exactly  so isclose() is the way to go.
Chris
[1] https://www.python.org/dev/peps/pep0008/#id10
 ~Ethan~
Pythonideas mailing list Pythonideas@python.org https://mail.python.org/mailman/listinfo/pythonideas Code of Conduct: http://python.org/psf/codeofconduct/
Chris Barker chris.barker32lpuo7BZBA@public.gmane.org writes:
Modulo error checking, etc, the function will return the result of::
abs(ab) <= max( rel_tolerance * min(abs(a), abs(b), abs_tolerance )
I think you're missing a closing parenthesis here.
Best, Nikolaus
Chris Barker chris.barker32lpuo7BZBA@public.gmane.org writes:
``abs(ab) <= tol*abs(a)``
``abs(ab) <= tol * max( abs(a), abs(b) )``
``abs(ab) <= tol * min( abs(a), abs(b) )``
``abs(ab) <= tol * (a + b)/2``
NOTE: (2) and (3) can also be written as:
 ``(abs(ab) <= tol*abs(a)) or (abs(ab) <= tol*abs(a))``
I think the last 'a' should be a 'b'?
 ``(abs(ab) <= tol*abs(a)) and (abs(ab) <= tol*abs(a))``
Same here?
Best, Nikolaus
I meant for this not to send... LOL.
But since it did..
``abs(ab) <= tol * max( abs(a), abs(b) )``
``abs(ab) <= tol * min( abs(a), abs(b) )``
 ``(abs(ab) <= tol*abs(a)) or (abs(ab) <= tol*abs(a))``
I think the last 'a' should be a 'b'?
 ``(abs(ab) <= tol*abs(a)) and (abs(ab) <= tol*abs(a))``
Same here?
Yes, I think they both should be 'b'. Otherwise both sides are the same.
Ron
First, +0.5 overall. The only real downside is that it takes math further away from C math.h, which is not much of a downside.
And now, the nits:
On Thursday, February 5, 2015 9:14 AM, Chris Barker chris.barker@noaa.gov wrote:
Nonfloat types
The primary usecase is expected to be floating point numbers. However, users may want to compare other numeric types similarly. In theory, it should work for any type that supports ``abs()``, comparisons, and subtraction. The code will be written and tested to
accommodate these types:
Surely the type also has to support multiplication (or division, if you choose to implement it that way) as well, right?
Also, are you sure your implementation doesn't need any explicit isinf/isnan checks? You mentioned something about cmath.isnan for complex numbers.
Also, what types like datetime, which support subtraction, but the result of subtraction is a different type, and only the latter supports abs and multiplication? An isclose method can be written that handles those types properly, but one can also be written that doesn't. I don't think it's important to support it (IIRC, numpy.isclose doesn't work on datetimes...), but it might be worth mentioning that, as a consequence of the exact rule being used, datetimes won't work (possibly even saying "just as with np.isclose", if I'm remembering right).
unittest assertion
Presumably this is just copying assertAlmostEqual and replacing the element comparisons with a call to isclose, passing the params along?
How much difference does it make?
This section relies on transformations that are valid for reals but not for floats. In particular:
delta = tol * (ab)
or::
delta / tol = (ab)
If a and b are subnormals, the multiplication could underflow to 0, but the division won't; if delta is a very large number, the division could overflow to inf but the multiplication won't. (I'm assuming delta < 1.0, of course.) Your later substitution has the same issue. I think your ultimate conclusion is right, but your proof doesn't really prove it unless you take the extreme cases into account.
The case that uses the arithmetic mean of the two values requires that the value be either added together before dividing by 2, which could result in extra overflow to inf for very large numbers, or require each value to be divided by two before being added together, which
could result in underflow to inf for very small numbers.
Dividing by two results in underflow to 0, not inf. (Adding before dividing can result in overflow to inf just as it can to inf, of course, but I don't think that needs mentioning.)
Also, dividing by two only results in underflow to 0 for two numbers, +/5e324, and I don't think there's any case where that underflow can cause a problem where you wouldn't already underflow to 0 unless tol >= 1.0, so I'm not sure this is actually a problem to worry about. That being said, your final conclusion is again hard to argue with: the benefit is so small it's not really worth _any_ significant cost, even the cost of having to explain why there's no cost. :)
Relative Tolerance Default
This section depends on the fact that a Python float has about 1e16 precision. But technically, that isn't true; it has about sys.float_info.epsilon precision, which is up to the implementation; CPython leaves it up to the C implementation's double type, and C89 leaves that up to the platform. That's why sys.float_info.epsilon is available in the first placebecause you can't know it a priori.
In practice, I don't know of any modern platforms that don't use IEEE double or something close enough, so it's always going to be 2.2e16ish. And I don't think you want the default tolerance to be platformdependent. And I don't think you want to put qualifiers in all the halfdozen places you mention the 1e9. I just think it's worth mentioning in a parenthetical somewhere in this paragraph, like maybe:
The relative tolerance required for two values to be considered "close" is entirely usecase dependent. Nevertheless, the relative tolerance needs to be less than 1.0, and greater than 1e16
(approximate precision of a python float *on almost all platforms*). Absolute tolerance default
The absolute tolerance value will be used primarily for comparing to zero. The absolute tolerance required to determine if a value is "close" to zero is entirely usecase dependent. There is also essentially no bounds to the useful range  expected values would conceivably be anywhere within the limits of a python float. Thus a
default of 0.0 is selected.
Does that default work properly for the other supported types? Or do you need to use the defaultconstructed or zeroconstructed value for some appropriate type? (Sorry for C++ terminology, but you know what I meanand the C++ rules do work for these Python types.) Maybe it's better to just specify "zero" and leave it up to the implementation to do the trivial thing if possible or something more complicated if necessary, rather than specifying that it's definitely the float 0.0?
Expected Uses
The primary expected use case is various forms of testing  "are the results computed near what I expect as a result?" This sort of test may or may not be part of a formal unit testing suite. Such testing could be used oneoff at the command line, in an iPython notebook, part of doctests, or simple assets in an ``if __name__ == "__main__"`` block.
Typo: "...or simple *asserts* in an..."
The proposed unitest.TestCase assertion would have course be used in
unit testing.
Typo: "...of course...".
On Thu, Feb 5, 2015 at 2:46 PM, Andrew Barnert abarnert@yahoo.com wrote:
Nonfloat types
The primary usecase is expected to be floating point numbers. However, users may want to compare other numeric types similarly. In theory, it should work for any type that supports ``abs()``, comparisons, and subtraction. The code will be written and tested to
accommodate these types:
Surely the type also has to support multiplication (or division, if you choose to implement it that way) as well, right?
yes, multiplication, not sure how I missed that. No division necessary.
Also, are you sure your implementation doesn't need any explicit isinf/isnan checks? You mentioned something about cmath.isnan for complex numbers.
yeah, there's that. Though it seems to work fine for Fraction and Decimal so far  I'm taking a TDD approach  if it works with a test, I don't worry about it ;)
This brings up a bigger issue, that I'm pretty nuetral on:
What to do about ducktyping and various types it may get called with? My intent was to count of duck typing completely:
1) I'll test with the types I mentioned in the PEP (I need a few more tests to be rigorous)
2) For any other type, it may work, it may not, and hopefully users will get an Exception, rather than a spurious result if it doesn't.
I can't possible test with all types, so what else to do?
Would datetime.timedetla be worth explicitly supporting? the concept of a two datetimes being relatively close doesn't make any sense  relative to what???? And abs() fails for datetime already. I'll check timedelta  it should work, I think  abs() does anyway.
I'm a bit on the fence as to whether to do type dispatch for Decimal, as well  there are some oddities that may require it.
Presumably this is just copying assertAlmostEqual and replacing the element comparisons with a call to isclose, passing the params along?
yup  but worth it? Maybe not.
How much difference does it make?
This section relies on transformations that are valid for reals but not for floats. In particular:
does this effect the result? I also assumed a and b are positive and a>b, just to make all the math easier (and easier to write). I'm not really trying to do a proper rigorous formal proof here. And it does check out computationally  at least with comparing to 1.0, there is no floating point value that passes some of the test, but not others with a rel_tol of 1e9 (there is one value that does for 1e8)
delta = tol * (ab)
or::
delta / tol = (ab)
If a and b are subnormals, the multiplication could underflow to 0, but the
division won't;
there is no division in the final computation anyway  I only understand enough FP to barely pass a class with Kahan, but I think it's OK to do the derivation with real numbers, and then check the FP consequences with the actual calculation in this case.
if delta is a very large number, the division could overflow to inf but the
multiplication won't. (I'm assuming delta < 1.0, of course.) Your later substitution has the same issue. I think your ultimate conclusion is right, but your proof doesn't really prove it unless you take the extreme cases into account.
I"m not sure all that needs to be in the PEP anyway  it's jsut that thre was a lot of discussion about the various methods, and I had handwavingly said "it doesn't matter for small tolerances"  so this is simply making that point for formally.
The case that uses the arithmetic mean of the two values requires that
the value be either added together before dividing by 2, which could result in extra overflow to inf for very large numbers, or require each value to be divided by two before being added together, which
could result in underflow to inf for very small numbers.
Dividing by two results in underflow to 0, not inf.
oops, duh. thanks.
Also, dividing by two only results in underflow to 0 for two numbers, +/5e324, and I don't think there's any case where that underflow can cause a problem where you wouldn't already underflow to 0 unless tol >= 1.0, so I'm not sure this is actually a problem to worry about.
no  not really  being a bit pedantic ;)
Relative Tolerance Default
This section depends on the fact that a Python float has about 1e16 precision.
indeed.
But technically, that isn't true; it has about sys.float_info.epsilon precision, which is up to the implementation; CPython leaves it up to the C implementation's double type, and C89 leaves that up to the platform. That's why sys.float_info.epsilon is available in the first placebecause you can't know it a priori.
In practice, I don't know of any modern platforms that don't use IEEE double or something close enough, so it's always going to be 2.2e16ish.
that's what I was assuming  but I guess there is always micropython  does it have machine doubles? I assume the common mobile platfroms do  but what do I know?
And I don't think you want the default tolerance to be platformdependent. And I don't think you want to put qualifiers in all the halfdozen places you mention the 1e9. I just think it's worth mentioning in a parenthetical somewhere in this paragraph.
Good idea.
default of 0.0 is selected.
Does that default work properly for the other supported types?
see above for the ducktyping question  it does work for all the types I've tested with.
Maybe it's better to just specify "zero" and leave it up to the implementation to do the trivial thing if possible or something more complicated if necessary, rather than specifying that it's definitely the float 0.0?
done in the PEP.
Thanks for the typo fixes.
Chris
On Feb 5, 2015, at 16:53, Chris Barker chris.barker@noaa.gov wrote:
On Thu, Feb 5, 2015 at 2:46 PM, Andrew Barnert abarnert@yahoo.com wrote:
Nonfloat types
The primary usecase is expected to be floating point numbers. However, users may want to compare other numeric types similarly. In theory, it should work for any type that supports ``abs()``, comparisons, and subtraction. The code will be written and tested to
accommodate these types:
Surely the type also has to support multiplication (or division, if you choose to implement it that way) as well, right?
yes, multiplication, not sure how I missed that. No division necessary.
Also, are you sure your implementation doesn't need any explicit isinf/isnan checks? You mentioned something about cmath.isnan for complex numbers.
yeah, there's that. Though it seems to work fine for Fraction and Decimal so far  I'm taking a TDD approach  if it works with a test, I don't worry about it ;)
Well, it would be nice if the docs could list the requirements on the type. It should be pretty simple to get them just by instrumenting a numeric type and seeing what gets called, if it isn't obvious enough from the code.
It would be even nicer if it happened to be a subset of the methods of numbers.Complex, so you could just say any Complex or Complexlike type (IIRC, Decimal doesn't subclass that because it doesn't quite qualify?) and most people could stop reading there. (Maybe you could even use that as the static type in the annotations/stub?) But if that isn't possible, that's fine.
This brings up a bigger issue, that I'm pretty nuetral on:
What to do about ducktyping and various types it may get called with? My intent was to count of duck typing completely:
I think that makes senseif it's possible, do that; if you have to add any special casing for any important type, I guess you have to, but hopefully you don't.
Calling math.isnan(abs(x)) instead of math.isnan(x) should work with any Complex, and with Decimal.
Or you can even try isnan and on TypeError assume it's not nan, and then it'll work on "numberish" types too.
The zeroconstruction issue might also show up with some numberish types, but I'd guess that most types that have __float__ also have a constructor from float, so that probably won't hurt much.
The other requirements (that __sub__ and __mul__ are sensible, __abs__ returns a noncomplexish version of the same type, etc.) all seem like they shouldn't be a problem at all for anything at all numberish.
I can't possible test with all types, so what else to do?
You can build a custom stupid numeric type that only implements exactly what you claim to require. That's good evidence your duck typing and your documentation match.
Also, it might be worth testing what happens with, say, a numpy array. That seems be a good test of the "it works it raises, for any reasonable type". (Testing a numpy matrix might be a further useful test, except I'm not sure matrix counts as a reasonable type.)
Would datetime.timedetla be worth explicitly supporting?
I don't think it's important enough. If you _implicitly_ support it through duck typing, that's cool (and a good sign that it'll work on many reasonable thirdparty types), but I wouldn't put extra work into it, especially not special casing, especially if you don't need any special casing for the important types.
the concept of a two datetimes being relatively close doesn't make any sense  relative to what????
Well, _absolute_ tolerance for datetimes makes sense (I've even used it in real code...), but since isclose is primarily a relative tolerance function, yeah, definitely don't try to make that work.
And abs() fails for datetime already. I'll check timedelta  it should work, I think  abs() does anyway.
But I'm pretty sure math.isnan(abs(x)) doesn't. That's exactly what the try/except could solve, if you want to solve it.
How much difference does it make?
This section relies on transformations that are valid for reals but not for floats. In particular:
does this effect the result? I also assumed a and b are positive and a>b, just to make all the math easier (and easier to write). I'm not really trying to do a proper rigorous formal proof here. And it does check out computationally  at least with comparing to 1.0,
Well, that's the whole pointwhen your values are all around 1.0, float issues are a whole lot simpler. Does it work for extremal and nearextremal values and values about 1e9 away from the extremes? If you're trying to make the point (semi)formally by deriving based on reals and then testing for rounding, underflow, and overflow, that's what you need to test on.
On the other hand, I think (as you hint later) that it might be better just to not try to make the point formally in the PEP.
On 02/06/2015 12:12 AM, Andrew Barnert wrote:
On the other hand, I think (as you hint later) that it might be better just to not try to make the point formally in the PEP.
And it's a whole lot simpler to write...
if isclose(x, y): ...
instead of...
if y  y * 1e8 <= x <= y + y * 1e8: ...
I think the isclose() line is much easier to read!
And the function will probably get it right, where I may get one of the signs reversed, or make some other silly error in the expression.
So +1 from me.
Cheers, Ron
On Wed, 4 Feb 2015 16:48:41 0800 Chris Barker chris.barker@noaa.gov wrote:
Hi folks,
Time for Take 2 (or is that take 200?)
No, I haven't dropped this ;) After the lengthy thread, I went through and tried to take into account all the thoughts and comments. The result is a PEP with a bunch more explanation, and some slightly different decisions.
I think it is a nuisance that this PEP doesn't work at zero by default. If this is meant to ease the life of nonspecialist users then it should do the right thing out of the box, IMO.
Regards
Antoine.
On Thu, Feb 5, 2015 at 3:04 PM, Antoine Pitrou solipsis@pitrou.net wrote:
I think it is a nuisance that this PEP doesn't work at zero by default.
I think this is the biggest point of contention  it' s a hard problem.
If this is meant to ease the life of nonspecialist users then it
should do the right thing out of the box, IMO.
That's the trick  there simply IS NO "right thing" to be done out of the box. A relative comparison to zero is mathematically and computationally impossible  you need an absolute comparison. And to set a default for that you need to know the general magnitude of the values your user is going to have  and we can't know that.
If we really believe that almost all people will be working with numbers of around a magnitude of 1, then we could set a default  by why in the world are you using floating point numbers with a range of about 1e300 to 1e300 in that case?
NOTE: numpy.allclose() uses an absolute tolerance default of 1e08  so someone thought that was a good idea. But as I've been thinking about all this, I decided it was very dangerous. In fact, I just did a quick grep of the unit tests in my current code base: 159 uses of allclose, and almost none with an absolute tolerance specified  now I need to go and check those, most of them probably need a more carefully thought out value.
I'm (and my team) are just one careless programmer, I guess, but I don't think I'm the only one that tends to use defaults, and only go back and think about if a test fails. And as you say, the "nonspecialist users" are the most likely to be careless.
Chris
On Fri, Feb 6, 2015 at 11:24 AM, Chris Barker chris.barker@noaa.gov wrote:
If we really believe that almost all people will be working with numbers of around a magnitude of 1, then we could set a default  by why in the world are you using floating point numbers with a range of about 1e300 to 1e300 in that case?
Because 'float' is the one obvious way to handle nonintegral numbers in Python. If you simply put "x = 1.1" in your code, you get a float, not a decimal.Decimal, not a fractions.Fraction, and not a complex. We don't need separate data types for "numbers around 1" and "numbers around 1e100"... not for the builtin float type, anyway. To the greatest extent possible, stuff should "just work".
Unfortunately the abstraction will always leak, especially when you work with numbers > 2**53 or < 2**53, but certainly there's nothing wrong with using a data type that gives you *more* range. In theory, a future version of Python should be able to switch to IEEE 754 Quad Precision and push the boundaries out, without breaking anyone's code. Then people will be using a data type with a range of something like 1e5000 to 1e5000... to store values close to 1. Ain't computing awesome! (Obligatory XKCD link: http://xkcd.com/676/ )
ChrisA
On Thu, Feb 5, 2015 at 4:40 PM, Chris Angelico rosuav@gmail.com wrote:
If we really believe that almost all people will be working with numbers
of
around a magnitude of 1, then we could set a default  by why in the
world
are you using floating point numbers with a range of about 1e300 to
1e300
in that case?
Because 'float' is the one obvious way to handle nonintegral numbers in Python. If you simply put "x = 1.1" in your code, you get a float,
Fair enough  more to the point  we HAVE floats because we want folks to be able to use a wide range of values  once we have that, we can't count on users generally only using a small part of that range.
Chris
On Fri, Feb 6, 2015 at 11:56 AM, Chris Barker chris.barker@noaa.gov wrote:
On Thu, Feb 5, 2015 at 4:40 PM, Chris Angelico rosuav@gmail.com wrote:
If we really believe that almost all people will be working with numbers of around a magnitude of 1, then we could set a default  by why in the world are you using floating point numbers with a range of about 1e300 to 1e300 in that case?
Because 'float' is the one obvious way to handle nonintegral numbers in Python. If you simply put "x = 1.1" in your code, you get a float,
Fair enough  more to the point  we HAVE floats because we want folks to be able to use a wide range of values  once we have that, we can't count on users generally only using a small part of that range.
Right. So crafting default tolerances for any particular "expected range" isn't particularly safe.
+1 for the current behaviour of not specifying absolute tolerance.
ChrisA
On Thu, Feb 5, 2015 at 4:24 PM, Chris Barker chris.barker@noaa.gov wrote:
If we really believe that almost all people will be working with numbers of around a magnitude of 1, then we could set a default  by why in the world are you using floating point numbers with a range of about 1e300 to 1e300 in that case?
It's an observable fact, though, that people generally do take care to ensure that their numbers are within a few orders of magnitude of 1, e.g. by choosing appropriate units, logtransforming, etc.
You could just as reasonably ask why in the world SI prefixes exist when we have access to scientific notation. And I wouldn't know the answer :). But they do!
n
Nathaniel Smith njse+AXbWqSrlAAvxtiuMwx3w@public.gmane.org writes:
It's an observable fact, though, that people generally do take care to ensure that their numbers are within a few orders of magnitude of 1, e.g. by choosing appropriate units, logtransforming, etc.
Do you have a credible source for that, or are you just speculating? At least in my field I'm regularly working with e.g. particle densities in the order of 10^19, and I've never seen anyone logtransforming these numbers (let alone inventing new units for them).
You could just as reasonably ask why in the world SI prefixes exist when we have access to scientific notation. And I wouldn't know the answer :). But they do!
Because they are shorter to write, and easier to parse. Compare
3.12 x 10^6 s
(13 characters, not counting ^) with
3.12 µs
(7 characters).
HTH, Nikolaus
Chris Barker chris.barker32lpuo7BZBA@public.gmane.org writes:
On Thu, Feb 5, 2015 at 3:04 PM, Antoine Pitrou solipsisxNDA5Wrcr86sTnJN9+BGXg@public.gmane.org wrote:
I think it is a nuisance that this PEP doesn't work at zero by default.
I think this is the biggest point of contention  it' s a hard problem.
I think the PEP does a very good job of explaining why a default absolute tolerance does not make sense, and why 1e8 is a reasonable default for relative tolerance.
Until someone makes an at least equally good case for a nonzero default absolute tolerance (and "too bad it doesn't work when testing against zero" isn't one), there really isn't much to discuss (IHMO).
Best, Nikolaus
Ok, more simply then: does is_close_to(0.0, 0.0) return True?
On Thu, 5 Feb 2015 16:24:21 0800 Chris Barker chris.barker@noaa.gov wrote:
On Thu, Feb 5, 2015 at 3:04 PM, Antoine Pitrou solipsis@pitrou.net wrote:
I think it is a nuisance that this PEP doesn't work at zero by default.
I think this is the biggest point of contention  it' s a hard problem.
If this is meant to ease the life of nonspecialist users then it
should do the right thing out of the box, IMO.
That's the trick  there simply IS NO "right thing" to be done out of the box. A relative comparison to zero is mathematically and computationally impossible  you need an absolute comparison. And to set a default for that you need to know the general magnitude of the values your user is going to have  and we can't know that.
If we really believe that almost all people will be working with numbers of around a magnitude of 1, then we could set a default  by why in the world are you using floating point numbers with a range of about 1e300 to 1e300 in that case?
NOTE: numpy.allclose() uses an absolute tolerance default of 1e08  so someone thought that was a good idea. But as I've been thinking about all this, I decided it was very dangerous. In fact, I just did a quick grep of the unit tests in my current code base: 159 uses of allclose, and almost none with an absolute tolerance specified  now I need to go and check those, most of them probably need a more carefully thought out value.
I'm (and my team) are just one careless programmer, I guess, but I don't think I'm the only one that tends to use defaults, and only go back and think about if a test fails. And as you say, the "nonspecialist users" are the most likely to be careless.
Chris
On 6 February 2015 at 12:28, Antoine Pitrou solipsis@pitrou.net wrote:
Ok, more simply then: does is_close_to(0.0, 0.0) return True?
From the formula in the PEP
"""abs(ab) <= max( rel_tolerance * min(abs(a), abs(b), abs_tolerance )"""
yes it does. More generally, is_close_to(x, x) is always true. That's a key requirement  that "closeness" includes equality.
I think the "weirdness" around zero is simply that there's no x for which is_close_to(x, 0) and x != 0. Which TBH isn't really all that weird :)
FWIW, I'm +0.5 with the PEP. (It lost a 0.5 for no other reason than I don't actually have a use for the function, so it's mostly theoretical interest for me).
Paul
On 6 February 2015 at 23:35, Paul Moore p.f.moore@gmail.com wrote:
On 6 February 2015 at 12:28, Antoine Pitrou solipsis@pitrou.net wrote:
Ok, more simply then: does is_close_to(0.0, 0.0) return True?
From the formula in the PEP
"""abs(ab) <= max( rel_tolerance * min(abs(a), abs(b), abs_tolerance )"""
yes it does. More generally, is_close_to(x, x) is always true. That's a key requirement  that "closeness" includes equality.
I think the "weirdness" around zero is simply that there's no x for which is_close_to(x, 0) and x != 0. Which TBH isn't really all that weird :)
Right, by default it degrades to exact equality when one of the operands is zero, just as it will degrade to exact equality when the relative tolerance is set to zero. This should probably be stated explicitly in the PEP, since it isn't immediately obvious from the formal definition.
+1 on the PEP from me  it meets the goal I suggested of being clearly better for comparing floating point values than standard equality, and manages to hide most of the surprising complexity of floating point comparisons.
Regards, Nick.
Right, by default it degrades to exact equality when one of the operands is zero, just as it will degrade to exact equality when the relative tolerance is set to zero. This should probably be stated explicitly in the PEP,
Will do.
 Chris
There seems to be a typo in the PEP:
""" The new function will have the following signature:
is_close(a, b, rel_tolerance=1e9, abs_tolerance=0.0)
a and b : are the two values to be tested to relative closeness
rel_tolerance : is the relative tolerance  it is the amount of error allowed, relative to the magnitude a and b. For example, to set a tolerance of 5%, pass tol=0.05. The default tolerance is 1e8, which assures that the two values are the same within about 8 decimal digits.
abs_tolerance : is an minimum absolute tolerance level  useful for comparisons near zero. """
Note that the description says 1e8 whereas the signature says 1e9.
The github link in the PEP gives a 404:
https://github.com/PythonCHB/close_pep/blob/master
In general, I think the PEP approaches the problem from a slightly different angle than one might expect.
I don't really want to continue the bike shedding around the PEP, but just add a slightly different perspective which might be useful as additional comment in the PEP.
In practice, you'd probably want to check whether the result of a computation (the approximation) is within an certain *precision range* around the true value, in other words, you're interested in either the absolute error of your calculation when assuming that the true value is 0.0 and the relative one otherwise.
Say your true value is v and the calculated one v_approx. The absolute error is abs(v  v_approx), the relative error abs(1  v_approx/v) (for v != 0).
This gives the following implementation:
""" _debug = False
def reasonably_close(v_approx, v, max_rel_error=1e8, max_abs_error=1e8):
""" Check whether the calculated value v_approx is within error range of the true value v.
If v == 0.0, an absolute error check is done using max_abs_error as valid error range.
For all other values, a relative error check is done using max_rel_error as basis.
This is done, since it is assumed that if you're calculating a value close to a value, you'd expect the calculation to be correct within a certain precision range.
""" if v == 0.0: # Use the absolute error as basis when checking calculations which # should return 0.0. abs_error = abs(v_approx  v) if _debug: print ('abs error = %s' % abs_error) return (abs_error <= max_abs_error)
# Use the relative error for all other true values. rel_error = abs(1  v_approx / v) if _debug: print ('rel error = %s' % rel_error) return (rel_error <= max_rel_error)
# Test equal values assert reasonably_close(0.0, 0.0) assert reasonably_close(1e8, 1e8) assert reasonably_close(1.0, 1.0)
# Test around 0.0 assert reasonably_close(1e8, 0.0) assert not reasonably_close(2e8, 0.0) assert reasonably_close(1e8, 1e8) assert not reasonably_close(2e8, 0.0)
# Test close to 0.0 assert reasonably_close(1e8 + 1e20, 1e8) assert reasonably_close(1e8 + 1e16, 1e8) assert not reasonably_close(1e8 + 2e16, 1e8)
# Test around 1.0 assert reasonably_close(1 + 1e8, 1.0) assert not reasonably_close(1 + 2e8, 1.0)
# Test large values assert reasonably_close(1e8 + 1, 1e8) assert not reasonably_close(1e8 + 2, 1e8) assert reasonably_close(1e9 + 1, 1e9) assert reasonably_close(1e9 + 2, 1e9) assert reasonably_close(1e9 + 10, 1e9) assert not reasonably_close(1e9 + 11, 1e9) assert reasonably_close(1e9, 0.0) assert reasonably_close(1 + 1e9, 1.0)
# Test negative values assert reasonably_close(1.0, 1.0) assert reasonably_close(1e8, 0.0) assert reasonably_close(1  1e8, 1.0) assert reasonably_close(1e9, 0.0) assert reasonably_close(1  1e9, 1.0)
# Test different signs assert not reasonably_close(0.0, 1.0) assert not reasonably_close(0.0, 1.0) assert not reasonably_close(1.0, 1.0) """
On Friday, February 6, 2015, M.A. Lemburg mal@egenix.com wrote:
There seems to be a typo in the PEP:
Got it  thanks.
The github link in the PEP gives a 404:
Must be the link for when you're logged in  this one will get you there:
https://github.com/PythonCHB/close_pep
I don't really want to continue the bike shedding around the PEP, but just add a slightly different perspective which might be useful as additional comment in the PEP.
Thanks  I'll see what I can add to the discussion.
Too bad you couldn't easily find the sample code  your code is essentially the "asymmetric test" in the sample. And with Nathanial Smith's suggestion of a zerotolerance. If the PEP isn't convincing enough as to why I didn't decide to go that way, then suggested text would be appreciated.
But in short  there is more than one way to do it  most of them would work fine for most usecases  I finally settled on what appeared to be the consensus for "least surprising".
Chris
On Sun, Feb 8, 2015 at 7:22 AM, Chris Barker chris.barker@noaa.gov wrote:
The github link in the PEP gives a 404:
Must be the link for when you're logged in  this one will get you there:
Or possibly you want to specify a file name:
https://github.com/PythonCHB/close_pep/blob/master/pep0485.txt
ChrisA
+1 on PEP 485 as currently written.
Maybe it's not perfect in every edge case, but per Nick's standard, it is significantly "less wrong" than 'a == b'.
On Fri, Feb 6, 2015 at 5:48 AM, Nick Coghlan ncoghlan@gmail.com wrote:
On 6 February 2015 at 23:35, Paul Moore p.f.moore@gmail.com wrote:
On 6 February 2015 at 12:28, Antoine Pitrou solipsis@pitrou.net wrote:
Ok, more simply then: does is_close_to(0.0, 0.0) return True?
From the formula in the PEP
"""abs(ab) <= max( rel_tolerance * min(abs(a), abs(b), abs_tolerance
)"""
yes it does. More generally, is_close_to(x, x) is always true. That's a key requirement  that "closeness" includes equality.
I think the "weirdness" around zero is simply that there's no x for which is_close_to(x, 0) and x != 0. Which TBH isn't really all that weird :)
Right, by default it degrades to exact equality when one of the operands is zero, just as it will degrade to exact equality when the relative tolerance is set to zero. This should probably be stated explicitly in the PEP, since it isn't immediately obvious from the formal definition.
+1 on the PEP from me  it meets the goal I suggested of being clearly better for comparing floating point values than standard equality, and manages to hide most of the surprising complexity of floating point comparisons.
Regards, Nick.
 Nick Coghlan  ncoghlan@gmail.com  Brisbane, Australia _______________________________________________ Pythonideas mailing list Pythonideas@python.org https://mail.python.org/mailman/listinfo/pythonideas Code of Conduct: http://python.org/psf/codeofconduct/
OK,
If I kept track correctly, we didn't get any 1s on the whole concept  thanks!
I went through all the comments and adjusted or clarified the PEP as best I could. The new version is in gitHub, along with a draft implementation and tests:
https://github.com/PythonCHB/close_pep
and should be up on the official site soon.
The draft code is in isclose.py and tests in test_isclose.py  the other files have more stuff in them fro experimenting with alternate methods.
As far as implementation is concerned, I'm not sure what to do with Decimal: simply cast to floats? or do typebased dispatch? ( Decimal won't autoconvert a float to decimal, so it doesn't "just work" with the default float tolerance. But that's an implementation detail.
So what's the next step?
Chris
Here's the latest version of the PEP
PEP: 485 Title: A Function for testing approximate equality Version: $Revision$ LastModified: $Date$ Author: Christopher Barker Chris.Barker@noaa.gov Status: Draft Type: Standards Track ContentType: text/xrst Created: 20Jan2015 PythonVersion: 3.5 PostHistory:
Abstract ========
This PEP proposes the addition of a function to the standard library that determines whether one value is approximately equal or "close" to another value.
Rationale =========
Floating point values contain limited precision, which results in their being unable to exactly represent some values, and for errors to accumulate with repeated computation. As a result, it is common advice to only use an equality comparison in very specific situations. Often a inequality comparison fits the bill, but there are times (often in testing) where the programmer wants to determine whether a computed value is "close" to an expected value, without requiring them to be exactly equal. This is common enough, particularly in testing, and not always obvious how to do it, so it would be useful addition to the standard library.
Existing Implementations 
The standard library includes the ``unittest.TestCase.assertAlmostEqual`` method, but it:
* Is buried in the unittest.TestCase class
* Is an assertion, so you can't use it as a general test at the command line, etc. (easily)
* Is an absolute difference test. Often the measure of difference requires, particularly for floating point numbers, a relative error, i.e. "Are these two values within x% of eachother?", rather than an absolute error. Particularly when the magnitude of the values is unknown a priori.
The numpy package has the ``allclose()`` and ``isclose()`` functions, but they are only available with numpy.
The statistics package tests include an implementation, used for its unit tests.
One can also find discussion and sample implementations on Stack Overflow and other help sites.
Many other nonpython systems provide such a test, including the Boost C++ library and the APL language [4]_.
These existing implementations indicate that this is a common need and not trivial to write oneself, making it a candidate for the standard library.
Proposed Implementation =======================
NOTE: this PEP is the result of extended discussions on the pythonideas list [1]_.
The new function will go into the math module, and have the following signature::
isclose(a, b, rel_tol=1e9, abs_tol=0.0)
``a`` and ``b``: are the two values to be tested to relative closeness
``rel_tol``: is the relative tolerance  it is the amount of error allowed, relative to the larger absolute value of a or b. For example, to set a tolerance of 5%, pass tol=0.05. The default tolerance is 1e9, which assures that the two values are the same within about 9 decimal digits. ``rel_tol`` must be greater than 0.0
``abs_tol``: is an minimum absolute tolerance level  useful for comparisons near zero.
Modulo error checking, etc, the function will return the result of::
abs(ab) <= max( rel_tol * max(abs(a), abs(b)), abs_tol )
The name, ``isclose``, is selected for consistency with the existing ``isnan`` and ``isinf``.
Handling of nonfinite numbers 
The IEEE 754 special values of NaN, inf, and inf will be handled according to IEEE rules. Specifically, NaN is not considered close to any other value, including NaN. inf and inf are only considered close to themselves.
Nonfloat types 
The primary usecase is expected to be floating point numbers. However, users may want to compare other numeric types similarly. In theory, it should work for any type that supports ``abs()``, multiplication, comparisons, and subtraction. The code will be written and tested to accommodate these types:
* ``Decimal``
* ``int``
* ``Fraction``
* ``complex``: for complex, the absolute value of the complex values will be used for scaling and comparison. If a complex tolerance is passed in, the absolute value will be used as the tolerance.
Behavior near zero 
Relative comparison is problematic if either value is zero. By definition, no value is small relative to zero. And computationally, if either value is zero, the difference is the absolute value of the other value, and the computed absolute tolerance will be ``rel_tol`` times that value. When ``rel_tol`` is less than one, the difference will never be less than the tolerance.
However, while mathematically correct, there are many use cases where a user will need to know if a computed value is "close" to zero. This calls for an absolute tolerance test. If the user needs to call this function inside a loop or comprehension, where some, but not all, of the expected values may be zero, it is important that both a relative tolerance and absolute tolerance can be tested for with a single function with a single set of parameters.
There is a similar issue if the two values to be compared straddle zero: if a is approximately equal to b, then a and b will never be computed as "close".
To handle this case, an optional parameter, ``abs_tol`` can be used to set a minimum tolerance used in the case of very small or zero computed relative tolerance. That is, the values will be always be considered close if the difference between them is less than ``abs_tol``
The default absolute tolerance value is set to zero because there is no value that is appropriate for the general case. It is impossible to know an appropriate value without knowing the likely values expected for a given use case. If all the values tested are on order of one, then a value of about 1e9 might be appropriate, but that would be far too large if expected values are on order of 1e9 or smaller.
Any nonzero default might result in user's tests passing totally inappropriately. If, on the other hand, a test against zero fails the first time with defaults, a user will be prompted to select an appropriate value for the problem at hand in order to get the test to pass.
NOTE: that the author of this PEP has resolved to go back over many of his tests that use the numpy ``allclose()`` function, which provides a default absolute tolerance, and make sure that the default value is appropriate.
If the user sets the rel_tol parameter to 0.0, then only the absolute tolerance will effect the result. While not the goal of the function, it does allow it to be used as a purely absolute tolerance check as well.
Implementation 
A sample implementation is available (as of Jan 22, 2015) on gitHub:
https://github.com/PythonCHB/close_pep/blob/master/is_close.py
This implementation has a flag that lets the user select which relative tolerance test to apply  this PEP does not suggest that that be retained, but rather than the weak test be selected.
There are also drafts of this PEP and test code, etc. there:
https://github.com/PythonCHB/close_pep
Relative Difference ===================
There are essentially two ways to think about how close two numbers are to eachother:
Absolute difference: simply ``abs(ab)``
Relative difference: ``abs(ab)/scale_factor`` [2]_.
The absolute difference is trivial enough that this proposal focuses on the relative difference.
Usually, the scale factor is some function of the values under consideration, for instance:
1) The absolute value of one of the input values
2) The maximum absolute value of the two
3) The minimum absolute value of the two.
4) The absolute value of the arithmetic mean of the two
These leads to the following possibilities for determining if two values, a and b, are close to each other.
1) ``abs(ab) <= tol*abs(a)``
2) ``abs(ab) <= tol * max( abs(a), abs(b) )``
3) ``abs(ab) <= tol * min( abs(a), abs(b) )``
4) ``abs(ab) <= tol * (a + b)/2``
NOTE: (2) and (3) can also be written as:
2) ``(abs(ab) <= abs(tol*a)) or (abs(ab) <= abs(tol*b))``
3) ``(abs(ab) <= abs(tol*a)) and (abs(ab) <= abs(tol*b))``
(Boost refers to these as the "weak" and "strong" formulations [3]_) These can be a tiny bit more computationally efficient, and thus are used in the example code.
Each of these formulations can lead to slightly different results. However, if the tolerance value is small, the differences are quite small. In fact, often less than available floating point precision.
How much difference does it make? 
When selecting a method to determine closeness, one might want to know how much of a difference it could make to use one test or the other  i.e. how many values are there (or what range of values) that will pass one test, but not the other.
The largest difference is between options (2) and (3) where the allowable absolute difference is scaled by either the larger or smaller of the values.
Define ``delta`` to be the difference between the allowable absolute tolerance defined by the larger value and that defined by the smaller value. That is, the amount that the two input values need to be different in order to get a different result from the two tests. ``tol`` is the relative tolerance value.
Assume that ``a`` is the larger value and that both ``a`` and ``b`` are positive, to make the analysis a bit easier. ``delta`` is therefore::
delta = tol * (ab)
or::
delta / tol = (ab)
The largest absolute difference that would pass the test: ``(ab)``, equals the tolerance times the larger value::
(ab) = tol * a
Substituting into the expression for delta::
delta / tol = tol * a
so::
delta = tol**2 * a
For example, for ``a = 10``, ``b = 9``, ``tol = 0.1`` (10%):
maximum tolerance ``tol * a == 0.1 * 10 == 1.0``
minimum tolerance ``tol * b == 0.1 * 9.0 == 0.9``
delta = ``(1.0  0.9) = 0.1`` or ``tol**2 * a = 0.1**2 * 10 = .1``
The absolute difference between the maximum and minimum tolerance tests in this case could be substantial. However, the primary use case for the proposed function is testing the results of computations. In that case a relative tolerance is likely to be selected of much smaller magnitude.
For example, a relative tolerance of ``1e8`` is about half the precision available in a python float. In that case, the difference between the two tests is ``1e8**2 * a`` or ``1e16 * a``, which is close to the limit of precision of a python float. If the relative tolerance is set to the proposed default of 1e9 (or smaller), the difference between the two tests will be lost to the limits of precision of floating point. That is, each of the four methods will yield exactly the same results for all values of a and b.
In addition, in common use, tolerances are defined to 1 significant figure  that is, 1e9 is specifying about 9 decimal digits of accuracy. So the difference between the various possible tests is well below the precision to which the tolerance is specified.
Symmetry 
A relative comparison can be either symmetric or nonsymmetric. For a symmetric algorithm:
``isclose(a,b)`` is always the same as ``isclose(b,a)``
If a relative closeness test uses only one of the values (such as (1) above), then the result is asymmetric, i.e. isclose(a,b) is not necessarily the same as isclose(b,a).
Which approach is most appropriate depends on what question is being asked. If the question is: "are these two numbers close to each other?", there is no obvious ordering, and a symmetric test is most appropriate.
However, if the question is: "Is the computed value within x% of this known value?", then it is appropriate to scale the tolerance to the known value, and an asymmetric test is most appropriate.
From the previous section, it is clear that either approach would
yield the same or similar results in the common use cases. In that case, the goal of this proposal is to provide a function that is least likely to produce surprising results.
The symmetric approach provide an appealing consistency  it mirrors the symmetry of equality, and is less likely to confuse people. A symmetric test also relieves the user of the need to think about the order in which to set the arguments. It was also pointed out that there may be some cases where the order of evaluation may not be well defined, for instance in the case of comparing a set of values all against each other.
There may be cases when a user does need to know that a value is within a particular range of a known value. In that case, it is easy enough to simply write the test directly::
if ab <= tol*a:
(assuming a > b in this case). There is little need to provide a function for this particular case.
This proposal uses a symmetric test.
Which symmetric test? 
There are three symmetric tests considered:
The case that uses the arithmetic mean of the two values requires that the value be either added together before dividing by 2, which could result in extra overflow to inf for very large numbers, or require each value to be divided by two before being added together, which could result in underflow to zero for very small numbers. This effect would only occur at the very limit of float values, but it was decided there was no benefit to the method worth reducing the range of functionality or adding the complexity of checking values to determine the order of computation.
This leaves the boost "weak" test (2) or using the smaller value to scale the tolerance, or the Boost "strong" (3) test, which uses the smaller of the values to scale the tolerance. For small tolerance, they yield the same result, but this proposal uses the boost "weak" test case: it is symmetric and provides a more useful result for very large tolerances.
Large tolerances 
The most common use case is expected to be small tolerances  on order of the default 1e9. However there may be use cases where a user wants to know if two fairly disparate values are within a particular range of each other: "is a within 200% (rel_tol = 2.0) of b? In this case, the string test would never indicate that two values are within that range of each other if one of them is zero. The strong case, however would use the larger (nonzero) value for the test, and thus return true if one value is zero. For example: is 0 within 200% of 10? 200% of ten is 20, so the range within 200% of ten is 10 to +30. Zero falls within that range, so it will return True.
Defaults ========
Default values are required for the relative and absolute tolerance.
Relative Tolerance Default 
The relative tolerance required for two values to be considered "close" is entirely usecase dependent. Nevertheless, the relative tolerance needs to be greater than 1e16 (approximate precision of a python float). The value of 1e9 was selected because it is the largest relative tolerance for which the various possible methods will yield the same result, and it is also about half of the precision available to a python float. In the general case, a good numerical algorithm is not expected to lose more than about half of available digits of accuracy, and if a much larger tolerance is acceptable, the user should be considering the proper value in that case. Thus 1e9 is expected to "just work" for many cases.
Absolute tolerance default 
The absolute tolerance value will be used primarily for comparing to zero. The absolute tolerance required to determine if a value is "close" to zero is entirely usecase dependent. There is also essentially no bounds to the useful range  expected values would conceivably be anywhere within the limits of a python float. Thus a default of 0.0 is selected.
If, for a given use case, a user needs to compare to zero, the test will be guaranteed to fail the first time, and the user can select an appropriate value.
It was suggested that comparing to zero is, in fact, a common use case (evidence suggest that the numpy functions are often used with zero). In this case, it would be desirable to have a "useful" default. Values around 1e8 were suggested, being about half of floating point precision for values of around value 1.
However, to quote The Zen: "In the face of ambiguity, refuse the temptation to guess." Guessing that users will most often be concerned with values close to 1.0 would lead to spurious passing tests when used with smaller values  this is potentially more damaging than requiring the user to thoughtfully select an appropriate value.
Expected Uses =============
The primary expected use case is various forms of testing  "are the results computed near what I expect as a result?" This sort of test may or may not be part of a formal unit testing suite. Such testing could be used oneoff at the command line, in an iPython notebook, part of doctests, or simple asserts in an ``if __name__ == "__main__"`` block.
It would also be an appropriate function to use for the termination criteria for a simple iterative solution to an implicit function::
guess = something while True: new_guess = implicit_function(guess, *args) if isclose(new_guess, guess): break guess = new_guess
Inappropriate uses 
One use case for floating point comparison is testing the accuracy of a numerical algorithm. However, in this case, the numerical analyst ideally would be doing careful error propagation analysis, and should understand exactly what to test for. It is also likely that ULP (Unit in the Last Place) comparison may be called for. While this function may prove useful in such situations, It is not intended to be used in that way.
Other Approaches ================
``unittest.TestCase.assertAlmostEqual`` 
( https://docs.python.org/3/library/unittest.html#unittest.TestCase.assertAlmo... )
Tests that values are approximately (or not approximately) equal by computing the difference, rounding to the given number of decimal places (default 7), and comparing to zero.
This method is purely an absolute tolerance test, and does not address the need for a relative tolerance test.
numpy ``isclose()`` 
http://docs.scipy.org/doc/numpydev/reference/generated/numpy.isclose.html
The numpy package provides the vectorized functions isclose() and allclose(), for similar use cases as this proposal:
``isclose(a, b, rtol=1e05, atol=1e08, equal_nan=False)``
Returns a boolean array where two arrays are elementwise equal within a tolerance.
The tolerance values are positive, typically very small numbers. The relative difference (rtol * abs(b)) and the absolute difference atol are added together to compare against the absolute difference between a and b
In this approach, the absolute and relative tolerance are added together, rather than the ``or`` method used in this proposal. This is computationally more simple, and if relative tolerance is larger than the absolute tolerance, then the addition will have no effect. However, if the absolute and relative tolerances are of similar magnitude, then the allowed difference will be about twice as large as expected.
This makes the function harder to understand, with no computational advantage in this context.
Even more critically, if the values passed in are small compared to the absolute tolerance, then the relative tolerance will be completely swamped, perhaps unexpectedly.
This is why, in this proposal, the absolute tolerance defaults to zero  the user will be required to choose a value appropriate for the values at hand.
Boost floatingpoint comparison 
The Boost project ( [3]_ ) provides a floating point comparison function. It is a symmetric approach, with both "weak" (larger of the two relative errors) and "strong" (smaller of the two relative errors) options. This proposal uses the Boost "weak" approach. There is no need to complicate the API by providing the option to select different methods when the results will be similar in most cases, and the user is unlikely to know which to select in any case.
Alternate Proposals 
A Recipe '''''''''
The primary alternate proposal was to not provide a standard library function at all, but rather, provide a recipe for users to refer to. This would have the advantage that the recipe could provide and explain the various options, and let the user select that which is most appropriate. However, that would require anyone needing such a test to, at the very least, copy the function into their code base, and select the comparison method to use.
``zero_tol`` ''''''''''''
One possibility was to provide a zero tolerance parameter, rather than the absolute tolerance parameter. This would be an absolute tolerance that would only be applied in the case of one of the arguments being exactly zero. This would have the advantage of retaining the full relative tolerance behavior for all nonzero values, while allowing tests against zero to work. However, it would also result in the potentially surprising result that a small value could be "close" to zero, but not "close" to an even smaller value. e.g., 1e10 is "close" to zero, but not "close" to 1e11.
No absolute tolerance '''''''''''''''''''''
Given the issues with comparing to zero, another possibility would have been to only provide a relative tolerance, and let comparison to zero fail. In this case, the user would need to do a simple absolute test: `abs(val) < zero_tol` in the case where the comparison involved zero.
However, this would not allow the same call to be used for a sequence of values, such as in a loop or comprehension. Making the function far less useful. It is noted that the default abs_tol=0.0 achieves the same effect if the default is not overridden.
Other tests ''''''''''''
The other tests considered are all discussed in the Relative Error section above.
References ==========
.. [1] Pythonideas list discussion threads
https://mail.python.org/pipermail/pythonideas/2015January/030947.html
https://mail.python.org/pipermail/pythonideas/2015January/031124.html
https://mail.python.org/pipermail/pythonideas/2015January/031313.html
.. [2] Wikipedia page on relative difference
http://en.wikipedia.org/wiki/Relative_change_and_difference
.. [3] Boost project floatingpoint comparison algorithms
http://www.boost.org/doc/libs/1_35_0/libs/test/doc/components/test_tools/flo...
.. [4] 1976. R. H. Lathwell. APL comparison tolerance. Proceedings of the eighth international conference on APL Pages 255  258
http://dl.acm.org/citation.cfm?doid=800114.803685
.. Bruce Dawson's discussion of floating point.
https://randomascii.wordpress.com/2012/02/25/comparingfloatingpointnumber...
Copyright =========
This document has been placed in the public domain.
.. Local Variables: mode: indentedtext indenttabsmode: nil sentenceenddoublespace: t fillcolumn: 70 coding: utf8
On Feb 10, 2015, at 21:15, Chris Barker chris.barker@noaa.gov wrote:
As far as implementation is concerned, I'm not sure what to do with Decimal: simply cast to floats? or do typebased dispatch? ( Decimal won't autoconvert a float to decimal, so it doesn't "just work" with the default float tolerance. But that's an implementation detail.
One more possibility:
Many of the functions in math don't work on Decimal. And Decimal has its own methods and functions for almost all of them. So you could add Decimal.isclose, and make math.isclose with Decimal values be a TypeError. And considering that a Decimal can handle ridiculous things like 1e9999 as a relative tolerance, maybe that's even a good thing, not an unfortunate fallback?
(I'll read the rest of it later, but even if you didn't address a single one of my comments, I'm still probably at least +0.5.)
On 02/10/2015 11:15 PM, Chris Barker wrote:
Large tolerances
The most common use case is expected to be small tolerances  on order of the default 1e9. However there may be use cases where a user wants to know if two fairly disparate values are within a particular range of each other: "is a within 200% (rel_tol = 2.0) of b? In this case, the string test would never
string > strong "the strong test would never"
indicate that two values are within that range of each other if one of them is zero. The strong case, however would use the larger (nonzero) value for the
strong > weak "The weak case, ..."
test, and thus return true if one value is zero. For example: is 0 within 200% of 10? 200% of ten is 20, so the range within 200% of ten is 10 to +30. Zero falls within that range, so it will return True.
Ron
Hi,
Do you propose a builtin function? I would prefer to put it in the math module.
You may add a new method to unittest.TestCase? It would show all parameters on error. .assertTrue(isclose(...)) doesn't help on error.
Victor
On Feb 14, 2015, at 5:26 AM, Victor Stinner victor.stinner@gmail.com wrote:
Do you propose a builtin function? I would prefer to put it in the math module.
Yes, the math module is the proposed place for it.
may add a new method to unittest.TestCase? It would show all parameters on error. .assertTrue(isclose(...)) doesn't help on error.
Good point  I'm not much OS a unittest user  if someone wants to a unittest method, I think that would be fine.
I did originally have that in the PEP, but it got distracting, so Guido suggested it be removed.
Chris
On 15 February 2015 at 02:10, Chris Barker  NOAA Federal chris.barker@noaa.gov wrote:
On Feb 14, 2015, at 5:26 AM, Victor Stinner victor.stinner@gmail.com wrote:
Do you propose a builtin function? I would prefer to put it in the math module.
Yes, the math module is the proposed place for it.
may add a new method to unittest.TestCase? It would show all parameters on error. .assertTrue(isclose(...)) doesn't help on error.
Good point  I'm not much OS a unittest user  if someone wants to a unittest method, I think that would be fine.
I did originally have that in the PEP, but it got distracting, so Guido suggested it be removed.
I don't think it needs to be in the PEP  assuming the PEP gets accepted, we can do a followup RFE for unittest.
Cheers, Nick.
Good point  I'm not much OS a unittest user  if someone wants to a
unittest method, I think that would be fine.
I don't think it needs to be in the PEP  assuming the PEP gets accepted, we can do a followup RFE for unittest.
Sounds good  thanks.
Chris
+1 to adding a new method on unittest.TestCase.
~ Ian Lee
On Sat, Feb 14, 2015 at 5:26 AM, Victor Stinner victor.stinner@gmail.com wrote:
Hi,
Do you propose a builtin function? I would prefer to put it in the math module.
You may add a new method to unittest.TestCase? It would show all parameters on error. .assertTrue(isclose(...)) doesn't help on error.
Victor
Pythonideas mailing list Pythonideas@python.org https://mail.python.org/mailman/listinfo/pythonideas Code of Conduct: http://python.org/psf/codeofconduct/
Doesn't Python already have this with unittest.TestCase.assertAlmostEqual https://docs.python.org/3.4/library/unittest.html#unittest.TestCase.assertAlmostEqual ?
On Sat, Feb 14, 2015 at 7:26 AM, Victor Stinner victor.stinner@gmail.com wrote:
Hi,
Do you propose a builtin function? I would prefer to put it in the math module.
You may add a new method to unittest.TestCase? It would show all parameters on error. .assertTrue(isclose(...)) doesn't help on error.
Victor
Pythonideas mailing list Pythonideas@python.org https://mail.python.org/mailman/listinfo/pythonideas Code of Conduct: http://python.org/psf/codeofconduct/
Yes, but that asserts if they're not close. This just returns a bool.
On Mon, Feb 16, 2015 at 5:57 PM, Ryan Gonzalez rymg19@gmail.com wrote:
Doesn't Python already have this with unittest.TestCase.assertAlmostEqual https://docs.python.org/3.4/library/unittest.html#unittest.TestCase.assertAlmostEqual ?
On Sat, Feb 14, 2015 at 7:26 AM, Victor Stinner victor.stinner@gmail.com wrote:
Hi,
Do you propose a builtin function? I would prefer to put it in the math module.
You may add a new method to unittest.TestCase? It would show all parameters on error. .assertTrue(isclose(...)) doesn't help on error.
Victor
On Mon, Feb 16, 2015 at 2:57 PM, Ryan Gonzalez rymg19@gmail.com wrote:
Doesn't Python already have this with unittest.TestCase.assertAlmostEqual https://docs.python.org/3.4/library/unittest.html#unittest.TestCase.assertAlmostEqual ?
Please read the PEP  assertAlmostEQual is an absolute tolerance test  not a relative tolerance.
Chris
On Sat, Feb 14, 2015 at 7:26 AM, Victor Stinner victor.stinner@gmail.com wrote:
Hi,
Do you propose a builtin function? I would prefer to put it in the math module.
You may add a new method to unittest.TestCase? It would show all parameters on error. .assertTrue(isclose(...)) doesn't help on error.
Victor
Pythonideas mailing list Pythonideas@python.org https://mail.python.org/mailman/listinfo/pythonideas Code of Conduct: http://python.org/psf/codeofconduct/
 Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdcSEGFAULT. Wait, I don't think that was nulterminated." Personal reality distortion fields are immune to contradictory evidence.  srean Check out my website: http://kirbyfan64.github.io/
On Feb 6, 2015, at 4:29 AM, Antoine Pitrou solipsis@pitrou.net wrote:
Ok, more simply then: does is_close_to(0.0, 0.0) return True?
Yes.
But isclose(0.0, something_greater_than_zero)
Will not.
Chris
On Thu, 5 Feb 2015 16:24:21 0800 Chris Barker chris.barker@noaa.gov wrote:
On Thu, Feb 5, 2015 at 3:04 PM, Antoine Pitrou solipsis@pitrou.net wrote:
I think it is a nuisance that this PEP doesn't work at zero by default.
I think this is the biggest point of contention  it' s a hard problem.
If this is meant to ease the life of nonspecialist users then it
should do the right thing out of the box, IMO.
That's the trick  there simply IS NO "right thing" to be done out of the box. A relative comparison to zero is mathematically and computationally impossible  you need an absolute comparison. And to set a default for that you need to know the general magnitude of the values your user is going to have  and we can't know that.
If we really believe that almost all people will be working with numbers of around a magnitude of 1, then we could set a default  by why in the world are you using floating point numbers with a range of about 1e300 to 1e300 in that case?
NOTE: numpy.allclose() uses an absolute tolerance default of 1e08  so someone thought that was a good idea. But as I've been thinking about all this, I decided it was very dangerous. In fact, I just did a quick grep of the unit tests in my current code base: 159 uses of allclose, and almost none with an absolute tolerance specified  now I need to go and check those, most of them probably need a more carefully thought out value.
I'm (and my team) are just one careless programmer, I guess, but I don't think I'm the only one that tends to use defaults, and only go back and think about if a test fails. And as you say, the "nonspecialist users" are the most likely to be careless.
Chris
Pythonideas mailing list Pythonideas@python.org https://mail.python.org/mailman/listinfo/pythonideas Code of Conduct: http://python.org/psf/codeofconduct/
:
On Wed, Feb 04, 2015 at 04:48:41PM 0800, Chris Barker wrote:
Please take a look and express your approval or disapproval (+1, +0, 0, 1, or whatever).
+1
[]z.
I'm +1 on the idea, +1 on the name, but 1 on symmetry.
I'm glad that you included "other approaches" (what people actually do in the field). However, your argument that "is it that hard to just type it in" — the whole point of this method is to give a selfcommenting line that describes what the reader means. Doesn't it make sense to say: is *my value* close to *some true value*? If you wanted a symmetric test in english, you might say "are these values close to each other"?
Best,
Neil
On Thursday, February 5, 2015 at 12:14:25 PM UTC5, Chris Barker  NOAA Federal wrote:
Hi folks,
Time for Take 2 (or is that take 200?)
No, I haven't dropped this ;) After the lengthy thread, I went through and tried to take into account all the thoughts and comments. The result is a PEP with a bunch more explanation, and some slightly different decisions.
TL;DR:
Please take a look and express your approval or disapproval (+1, +0, 0, 1, or whatever). Please keep in mind that this has been discussed a lot, so try to not to reiterate what's already been said. And I'd really like it if you only gave 1, thumbs down, etc, if you really think this is worse than not having anything at all, rather than you'd just prefer something a little different. Also, if you do think this version is worse than nothing, but a different choice or two could change that, then let us know what  maybe we can reach consensus on something slightly different.
Also keep in mid that the goal here (in the words of Nick Coghlan): """ the key requirement here should be "provide a binary float comparison function that is significantly less wrong than the current 'a == b'" """ No need to split hairs.
https://www.python.org/dev/peps/pep0485/
Full PEP below, and in gitHub (with example code, tests, etc.) here:
https://github.com/PythonCHB/close_pep
Full Detail:
Here are the big changes:
 Symmetric test. Essentially, whether you want the symmetric or
asymmetric test depends on exactly what question you are asking, but I think we need to pick one. It makes little to no difference for most use cases (see below) , but I was persuaded by:
 Principle of least surprise  equality is symmetric, shouldn't
"closeness" be too?
 People don't want to have to remember what order to put the arguments
even if it doesn't matter, you have to think about whether it matters. A symmetric test doesn't require that.
 There are times when the order of comparison is not known  for
example if the users wants to test a bunch of values all against eachother.
On the other hand  the asymmetric test is a better option only when you are specifically asking the question: is this (computed) value within a precise tolerance of another(known) value, and that tolerance is fairly large (i.e 1%  10%). While this was brought up on this thread, no one had an actual usecase like that. And is it really that hard to write:
known  computed <= 0.1* expected
NOTE: that my example code has a flag with which you can pick which test to use  I did that for experimentation, but I think we want a simple API here. As soon as you provide that option people need to concern themselves with what to use.
 Defaults: I set the default relative tolerance to 1e9  because that
is a little more than half the precision available in a Python float, and it's the largest tolerance for which ALL the possible tests result in exactly the same result for all possible float values (see the PEP for the math). The default for absolute tolerance is 0.0. Better for people to get a failed test with a check for zero right away than accidentally use a totally inappropriate value.
Contentious Issues
 The Symmetric vs. Asymmetric thing  some folks seemed to have string
ideas about that, but i hope we can all agree that something is better than nothing, and symmetric is probably the least surprising. And take a look at the math in teh PEP  for small tolerance, which is the likely case for most tests  it literally makes no difference at all which test is used.
 Default for abs_tol  or even whether to have it or not. This is a
tough one  there are a lot of usecases for people testing against zero, so it's a pity not to have it do something reasonable by default in that case. However, there really is no reasonable universal default.
As Guido said: """ For someone who for whatever reason is manipulating quantities that are in the range of 1e100, 1e12 is about as large as infinity. """ And testing against zero really requires a absolute tolerance test, which is not hard to simply write:
abs(my_val) <= some_tolerance
and is covered by the unittest assert already.
So why include an absolute_tolerance at all?  Because this is likely to be used inside a comprehension or in the a unittest sequence comparison, so it is very helpful to be able to test a bunch of values some of which may be zero, all with a single function with a single set of parameters. And a similar approach has proven to be useful for numpy and the statistics test module. And again, the default is abs_tol=0.0  if the user sticks with defaults, they won't get bitten.
 Also  not really contentious (I hope) but I left the details out about
how the unittest assertion would work. That's because I don't really use unittest  I'm hoping someone else will write that. If it comes down to it, I can do it  it will probably look a lot like the existing sequence comparing assertions.
OK  I think that's it.
Let's see if we can drive this home.
Chris
PEP: 485 Title: A Function for testing approximate equality Version: $Revision$ LastModified: $Date$ Author: Christopher Barker <Chris....@noaa.gov javascript:> Status: Draft Type: Standards Track ContentType: text/xrst Created: 20Jan2015 PythonVersion: 3.5 PostHistory:
Abstract
This PEP proposes the addition of a function to the standard library that determines whether one value is approximately equal or "close" to another value. It is also proposed that an assertion be added to the ``unittest.TestCase`` class to provide easy access for those using unittest for testing.
Rationale
Floating point values contain limited precision, which results in their being unable to exactly represent some values, and for error to accumulate with repeated computation. As a result, it is common advice to only use an equality comparison in very specific situations. Often a inequality comparison fits the bill, but there are times (often in testing) where the programmer wants to determine whether a computed value is "close" to an expected value, without requiring them to be exactly equal. This is common enough, particularly in testing, and not always obvious how to do it, so it would be useful addition to the standard library.
Existing Implementations
The standard library includes the ``unittest.TestCase.assertAlmostEqual`` method, but it:
Is buried in the unittest.TestCase class
Is an assertion, so you can't use it as a general test at the command line, etc. (easily)
Is an absolute difference test. Often the measure of difference requires, particularly for floating point numbers, a relative error, i.e "Are these two values within x% of eachother?", rather than an absolute error. Particularly when the magnatude of the values is unknown a priori.
The numpy package has the ``allclose()`` and ``isclose()`` functions, but they are only available with numpy.
The statistics package tests include an implementation, used for its unit tests.
One can also find discussion and sample implementations on Stack Overflow and other help sites.
Many other nonpython systems provide such a test, including the Boost C++ library and the APL language (reference?).
These existing implementations indicate that this is a common need and not trivial to write oneself, making it a candidate for the standard library.
Proposed Implementation
NOTE: this PEP is the result of an extended discussion on the pythonideas list [1]_.
The new function will have the following signature::
is_close(a, b, rel_tolerance=1e9, abs_tolerance=0.0)
``a`` and ``b``: are the two values to be tested to relative closeness
``rel_tolerance``: is the relative tolerance  it is the amount of error allowed, relative to the magnitude a and b. For example, to set a tolerance of 5%, pass tol=0.05. The default tolerance is 1e8, which assures that the two values are the same within about 8 decimal digits.
``abs_tolerance``: is an minimum absolute tolerance level  useful for comparisons near zero.
Modulo error checking, etc, the function will return the result of::
abs(ab) <= max( rel_tolerance * min(abs(a), abs(b), abs_tolerance )
Handling of nonfinite numbers
The IEEE 754 special values of NaN, inf, and inf will be handled according to IEEE rules. Specifically, NaN is not considered close to any other value, including NaN. inf and inf are only considered close to themselves.
Nonfloat types
The primary usecase is expected to be floating point numbers. However, users may want to compare other numeric types similarly. In theory, it should work for any type that supports ``abs()``, comparisons, and subtraction. The code will be written and tested to accommodate these types:
``Decimal``: for Decimal, the tolerance must be set to a Decimal type.
``int``
``Fraction``
``complex``: for complex, ``abs(z)`` will be used for scaling and comparison.
Behavior near zero
Relative comparison is problematic if either value is zero. By definition, no value is small relative to zero. And computationally, if either value is zero, the difference is the absolute value of the other value, and the computed absolute tolerance will be rel_tolerance times that value. reltolerance is always less than one, so the difference will never be less than the tolerance.
However, while mathematically correct, there are many use cases where a user will need to know if a computed value is "close" to zero. This calls for an absolute tolerance test. If the user needs to call this function inside a loop or comprehension, where some, but not all, of the expected values may be zero, it is important that both a relative tolerance and absolute tolerance can be tested for with a single function with a single set of parameters.
There is a similar issue if the two values to be compared straddle zero: if a is approximately equal to b, then a and b will never be computed as "close".
To handle this case, an optional parameter, ``abs_tolerance`` can be used to set a minimum tolerance used in the case of very small or zero computed absolute tolerance. That is, the values will be always be considered close if the difference between them is less than the abs_tolerance
The default absolute tolerance value is set to zero because there is no value that is appropriate for the general case. It is impossible to know an appropriate value without knowing the likely values expected for a given use case. If all the values tested are on order of one, then a value of about 1e8 might be appropriate, but that would be far too large if expected values are on order of 1e12 or smaller.
Any nonzero default might result in user's tests passing totally inappropriately. If, on the other hand a test against zero fails the first time with defaults, a user will be prompted to select an appropriate value for the problem at hand in order to get the test to pass.
NOTE: that the author of this PEP has resolved to go back over many of his tests that use the numpy ``all_close()`` function, which provides a default abs_tolerance, and make sure that the default value is appropriate.
If the user sets the rel_tolerance parameter to 0.0, then only the absolute tolerance will effect the result. While not the goal of the function, it does allow it to be used as a purely absolute tolerance check as well.
unittest assertion
[need text here]
implementation
A sample implementation is available (as of Jan 22, 2015) on gitHub:
https://github.com/PythonCHB/close_pep/blob/master
This implementation has a flag that lets the user select which relative tolerance test to apply  this PEP does not suggest that that be retained, but rather than the strong test be selected.
Relative Difference
There are essentially two ways to think about how close two numbers are to eachother:
Absolute difference: simply ``abs(ab)``
Relative difference: ``abs(ab)/scale_factor`` [2]_.
The absolute difference is trivial enough that this proposal focuses on the relative difference.
Usually, the scale factor is some function of the values under consideration, for instance:
The absolute value of one of the input values
The maximum absolute value of the two
The minimum absolute value of the two.
The absolute value of the arithmetic mean of the two
These lead to the following possibilities for determining if two values, a and b, are close to each other.
``abs(ab) <= tol*abs(a)``
``abs(ab) <= tol * max( abs(a), abs(b) )``
``abs(ab) <= tol * min( abs(a), abs(b) )``
``abs(ab) <= tol * (a + b)/2``
NOTE: (2) and (3) can also be written as:
``(abs(ab) <= tol*abs(a)) or (abs(ab) <= tol*abs(a))``
``(abs(ab) <= tol*abs(a)) and (abs(ab) <= tol*abs(a))``
(Boost refers to these as the "weak" and "strong" formulations [3]_) These can be a tiny bit more computationally efficient, and thus are used in the example code.
Each of these formulations can lead to slightly different results. However, if the tolerance value is small, the differences are quite small. In fact, often less than available floating point precision.
How much difference does it make?
When selecting a method to determine closeness, one might want to know how much of a difference it could make to use one test or the other  i.e. how many values are there (or what range of values) that will pass one test, but not the other.
The largest difference is between options (2) and (3) where the allowable absolute difference is scaled by either the larger or smaller of the values.
Define ``delta`` to be the difference between the allowable absolute tolerance defined by the larger value and that defined by the smaller value. That is, the amount that the two input values need to be different in order to get a different result from the two tests. ``tol`` is the relative tolerance value.
Assume that ``a`` is the larger value and that both ``a`` and ``b`` are positive, to make the analysis a bit easier. ``delta`` is therefore::
delta = tol * (ab)
or::
delta / tol = (ab)
The largest absolute difference that would pass the test: ``(ab)``, equals the tolerance times the larger value::
(ab) = tol * a
Substituting into the expression for delta::
delta / tol = tol * a
so::
delta = tol**2 * a
For example, for ``a = 10``, ``b = 9``, ``tol = 0.1`` (10%):
maximum tolerance ``tol * a == 0.1 * 10 == 1.0``
minimum tolerance ``tol * b == 0.1 * 9.0 == 0.9``
delta = ``(1.0  0.9) * 0.1 = 0.1`` or ``tol**2 * a = 0.1**2 * 10 = .01``
The absolute difference between the maximum and minimum tolerance tests in this case could be substantial. However, the primary use case for the proposed function is testing the results of computations. In that case a relative tolerance is likely to be selected of much smaller magnitude.
For example, a relative tolerance of ``1e8`` is about half the precision available in a python float. In that case, the difference between the two tests is ``1e8**2 * a`` or ``1e16 * a``, which is close to the limit of precision of a python float. If the relative tolerance is set to the proposed default of 1e9 (or smaller), the difference between the two tests will be lost to the limits of precision of floating point. That is, each of the four methods will yield exactly the same results for all values of a and b.
In addition, in common use, tolerances are defined to 1 significant figure  that is, 1e8 is specifying about 8 decimal digits of accuracy. So the difference between the various possible tests is well below the precision to which the tolerance is specified.
Symmetry
A relative comparison can be either symmetric or nonsymmetric. For a symmetric algorithm:
``is_close_to(a,b)`` is always the same as ``is_close_to(b,a)``
If a relative closeness test uses only one of the values (such as (1) above), then the result is asymmetric, i.e. is_close_to(a,b) is not necessarily the same as is_close_to(b,a).
Which approach is most appropriate depends on what question is being asked. If the question is: "are these two numbers close to each other?", there is no obvious ordering, and a symmetric test is most appropriate.
However, if the question is: "Is the computed value within x% of this known value?", then it is appropriate to scale the tolerance to the known value, and an asymmetric test is most appropriate.
From the previous section, it is clear that either approach would yield the same or similar results in the common use cases. In that case, the goal of this proposal is to provide a function that is least likely to produce surprising results.
The symmetric approach provide an appealing consistency  it mirrors the symmetry of equality, and is less likely to confuse people. A symmetric test also relieves the user of the need to think about the order in which to set the arguments. It was also pointed out that there may be some cases where the order of evaluation may not be well defined, for instance in the case of comparing a set of values all against each other.
There may be cases when a user does need to know that a value is within a particular range of a known value. In that case, it is easy enough to simply write the test directly::
if ab <= tol*a:
(assuming a > b in this case). There is little need to provide a function for this particular case.
This proposal uses a symmetric test.
Which symmetric test?
There are three symmetric tests considered:
The case that uses the arithmetic mean of the two values requires that the value be either added together before dividing by 2, which could result in extra overflow to inf for very large numbers, or require each value to be divided by two before being added together, which could result in underflow to inf for very small numbers. This effect would only occur at the very limit of float values, but it was decided there as no benefit to the method worth reducing the range of functionality.
This leaves the boost "weak" test (2) or using the smaller value to scale the tolerance, or the Boost "strong" (3) test, which uses the smaller of the values to scale the tolerance. For small tolerance, they yield the same result, but this proposal uses the boost "strong" test case: it is symmetric and provides a slightly stricter criteria for tolerance.
Defaults
Default values are required for the relative and absolute tolerance.
Relative Tolerance Default
The relative tolerance required for two values to be considered "close" is entirely usecase dependent. Nevertheless, the relative tolerance needs to be less than 1.0, and greater than 1e16 (approximate precision of a python float). The value of 1e9 was selected because it is the largest relative tolerance for which the various possible methods will yield the same result, and it is also about half of the precision available to a python float. In the general case, a good numerical algorithm is not expected to lose more than about half of available digits of accuracy, and if a much larger tolerance is acceptable, the user should be considering the proper value in that case. Thus 1e9 is expected to "just work" for many cases.
Absolute tolerance default
The absolute tolerance value will be used primarily for comparing to zero. The absolute tolerance required to determine if a value is "close" to zero is entirely usecase dependent. There is also essentially no bounds to the useful range  expected values would conceivably be anywhere within the limits of a python float. Thus a default of 0.0 is selected.
If, for a given use case, a user needs to compare to zero, the test will be guaranteed to fail the first time, and the user can select an appropriate value.
It was suggested that comparing to zero is, in fact, a common use case (evidence suggest that the numpy functions are often used with zero). In this case, it would be desirable to have a "useful" default. Values around 1e8 were suggested, being about half of floating point precision for values of around value 1.
However, to quote The Zen: "In the face of ambiguity, refuse the temptation to guess." Guessing that users will most often be concerned with values close to 1.0 would lead to spurious passing tests when used with smaller values  this is potentially more damaging than requiring the user to thoughtfully select an appropriate value.
Expected Uses
The primary expected use case is various forms of testing  "are the results computed near what I expect as a result?" This sort of test may or may not be part of a formal unit testing suite. Such testing could be used oneoff at the command line, in an iPython notebook, part of doctests, or simple assets in an ``if __name__ == "__main__"`` block.
The proposed unitest.TestCase assertion would have course be used in unit testing.
It would also be an appropriate function to use for the termination criteria for a simple iterative solution to an implicit function::
guess = something while True: new_guess = implicit_function(guess, *args) if is_close(new_guess, guess): break guess = new_guess
Inappropriate uses
One use case for floating point comparison is testing the accuracy of a numerical algorithm. However, in this case, the numerical analyst ideally would be doing careful error propagation analysis, and should understand exactly what to test for. It is also likely that ULP (Unit in the Last Place) comparison may be called for. While this function may prove useful in such situations, It is not intended to be used in that way.
Other Approaches
``unittest.TestCase.assertAlmostEqual``
( https://docs.python.org/3/library/unittest.html#unittest.TestCase.assertAlmo... )
Tests that values are approximately (or not approximately) equal by computing the difference, rounding to the given number of decimal places (default 7), and comparing to zero.
This method is purely an absolute tolerance test, and does not address the need for a relative tolerance test.
numpy ``is_close()``
http://docs.scipy.org/doc/numpydev/reference/generated/numpy.isclose.html
The numpy package provides the vectorized functions is_close() and all_close, for similar use cases as this proposal:
``isclose(a, b, rtol=1e05, atol=1e08, equal_nan=False)``
Returns a boolean array where two arrays are elementwise equal within a tolerance. The tolerance values are positive, typically very small numbers. The relative difference (rtol * abs(b)) and the absolute difference atol are added together to compare against the absolute difference between a and b
In this approach, the absolute and relative tolerance are added together, rather than the ``or`` method used in this proposal. This is computationally more simple, and if relative tolerance is larger than the absolute tolerance, then the addition will have no effect. However, if the absolute and relative tolerances are of similar magnitude, then the allowed difference will be about twice as large as expected.
This makes the function harder to understand, with no computational advantage in this context.
Even more critically, if the values passed in are small compared to the absolute tolerance, then the relative tolerance will be completely swamped, perhaps unexpectedly.
This is why, in this proposal, the absolute tolerance defaults to zero  the user will be required to choose a value appropriate for the values at hand.
Boost floatingpoint comparison
The Boost project ( [3]_ ) provides a floating point comparison function. Is is a symmetric approach, with both "weak" (larger of the two relative errors) and "strong" (smaller of the two relative errors) options. This proposal uses the Boost "strong" approach. There is no need to complicate the API by providing the option to select different methods when the results will be similar in most cases, and the user is unlikely to know which to select in any case.
Alternate Proposals
A Recipe '''''''''
The primary alternate proposal was to not provide a standard library function at all, but rather, provide a recipe for users to refer to. This would have the advantage that the recipe could provide and explain the various options, and let the user select that which is most appropriate. However, that would require anyone needing such a test to, at the very least, copy the function into their code base, and select the comparison method to use.
In addition, adding the function to the standard library allows it to be used in the ``unittest.TestCase.assertIsClose()`` method, providing a substantial convenience to those using unittest.
``zero_tol`` ''''''''''''
One possibility was to provide a zero tolerance parameter, rather than the absolute tolerance parameter. This would be an absolute tolerance that would only be applied in the case of one of the arguments being exactly zero. This would have the advantage of retaining the full relative tolerance behavior for all nonzero values, while allowing tests against zero to work. However, it would also result in the potentially surprising result that a small value could be "close" to zero, but not "close" to an even smaller value. e.g., 1e10 is "close" to zero, but not "close" to 1e11.
No absolute tolerance '''''''''''''''''''''
Given the issues with comparing to zero, another possibility would have been to only provide a relative tolerance, and let every comparison to zero fail. In this case, the user would need to do a simple absolute test: `abs(val) < zero_tol` in the case where the comparison involved zero.
However, this would not allow the same call to be used for a sequence of values, such as in a loop or comprehension, or in the ``TestCase.assertClose()`` method. Making the function far less useful. It is noted that the default abs_tolerance=0.0 achieves the same effect if the default is not overidden.
Other tests ''''''''''''
The other tests considered are all discussed in the Relative Error section above.
References
.. [1] Pythonideas list discussion threads
https://mail.python.org/pipermail/pythonideas/2015January/030947.html
https://mail.python.org/pipermail/pythonideas/2015January/031124.html
https://mail.python.org/pipermail/pythonideas/2015January/031313.html
.. [2] Wikipedia page on relative difference
http://en.wikipedia.org/wiki/Relative_change_and_difference
.. [3] Boost project floatingpoint comparison algorithms
http://www.boost.org/doc/libs/1_35_0/libs/test/doc/components/test_tools/flo...
.. Bruce Dawson's discussion of floating point.
https://randomascii.wordpress.com/2012/02/25/comparingfloatingpointnumber...
Copyright
This document has been placed in the public domain.
.. Local Variables: mode: indentedtext indenttabsmode: nil sentenceenddoublespace: t fillcolumn: 70 coding: utf8

Christopher Barker, Ph.D. Oceanographer
Emergency Response Division NOAA/NOS/OR&R (206) 5266959 voice 7600 Sand Point Way NE (206) 5266329 fax Seattle, WA 98115 (206) 5266317 main reception
Chris....@noaa.gov javascript:
On 02/15/2015 05:07 PM, Neil Girdhar wrote:
I'm +1 on the idea, +1 on the name, but 1 on symmetry.
I'm glad that you included "other approaches" (what people actually do in the field). However, your argument that "is it that hard to just type it in" — the whole point of this method is to give a selfcommenting line that describes what the reader means. Doesn't it make sense to say: is *my value* close to *some true value*? If you wanted a symmetric test in english, you might say "are these values close to each other"?
My understanding of the main use of the function is to ask if two values can be considered equivalent within some relative and/or absolute error amount.
About those "is it really hard to write?" comments in the PEP. They refer to only the absolute comparisons. But they aren't as easy to get right as most people think. Floating point is hard enough to understand correctly that only a very few bother to write correct comparisons. ie.. not using <=. >=. and == in floating point comparisons and adding an appropriate epsilon.
I think at this point what we need is to make sure we have some very good tests and examples that represent the various different uses in different circumstances. That may also answer any concerns any one still might have about the various choices.
Cheers, Ron
On Sun, Feb 15, 2015 at 2:07 PM, Neil Girdhar mistersheik@gmail.com wrote:
I'm +1 on the idea, +1 on the name, but 1 on symmetry.
I'm glad that you included "other approaches" (what people actually do in the field). However, your argument that "is it that hard to just type it in" — the whole point of this method is to give a selfcommenting line that describes what the reader means.
yes. sure.
Doesn't it make sense to say: is *my value* close to *some true value*? If you wanted a symmetric test in english, you might say "are these values close to each other"?
exactly  it also makes sense to ask "are these values close to each other" The problem is that you can't have one function that does both (without flags or optional parameters. After a lengthy discussion, I decided that the later was a bit more useful and less surprising.
And do read carefully  if your tolerance is small  it doesn't matter at all.
Chris
participants (21)

Andrew Barnert

Antoine Pitrou

Chris Angelico

Chris Barker

Chris Barker  NOAA Federal

David Mertz

Ethan Furman

Guido van Rossum

Ian Lee

M.A. Lemburg

Marco Buttu

Nathaniel Smith

Neil Girdhar

Nick Coghlan

Nikolaus Rath

Paul Moore

Ron Adam

Ryan Gonzalez

Steven D'Aprano

Victor Stinner

Zero Piraeus