On Sun, Jan 25, 2015 at 05:21:53PM -0800, Chris Barker wrote: [...]
But adding a relative tolerance to unittest makes a lot of sense -- would "assertCloseTo" sound entirely too much like assertAlmostEqual? I think it may be OK if the docs for each pointed to the other.
CloseTo assumes an asymetric test, which isn't a given :-) I prefer ApproxEqual, although given that it is confusingly similar to AlmostEqual, IsClose would be my second preference.
The actual fuzzy comparison itself is handled by a function approx_equal(x, y, tol, rel).
NOTE: the difference between this and the my current PEP version is that that absolute tolerance defaults to something other than zero (though it looks like it does default to zero for the assert method), and it is a symmetric test (what Boost calls the "strong" test)
- somebody other than me should review NumericTestCase.assertApproxEqual and check that it does nothing unreasonable;
Well it requires the tolerance values to be set on the instance, and they default to zero. So if we were to add this to unittest.TestCase, would you make those instance attributes of TestCase?
No, I would modify it to do something like this: if tol is None: tol = getattr(self, "tol", 0.0) # or some other default and similar for rel. I recommend using short names for the two error tolerances, tol and rel, because if people are going to be writing a lot of tests, having to write: self.assertIsClose(x, y, absolute_tolerance=0.001) will get tiresome.
- since there are considerable disagreements about the right way to handle a fuzzy comparison when *both* an absolute and relative error are given, people who disagree with the default definition can simply subclass TestCase and redefine the approx_equal method.
(Which is much simpler than having to write the whole assertApproxEqual method from scratch.)
what assertApproxEqual does is add the ability to test a while sequence of values -- much like numpy's all_close. Do any of the other TestCase assertions provide that?
I was motivated by assertEqual and the various sequence/list methods. I wanted to compare two lists element-wise using an approximate comparison, e.g.: # func1 and func2 are alternate implementations of the same thing a = [func1(x) for x in values] b = [func2(x) for x in values] self.assertApproxEqual(a, b) It wouldn't be meaningful to compare the two lists for approximate equality as lists, but it does make sense to do an element-by-element comparison. In the event of failure, you want an error message that is more specific than just the two lists being unequal.
But you could also add an optional parameter to pass in an alternate comparison function, rather than have it be a method of TestCase. As I said, I think it's better to have it available, and discoverable, for use outside of unitest.
That's an alternative too. I guess it boils down to whether you prefer inheritance or the strategy design pattern :-) I do think there are two distinct use-cases that should be included in the PEP: (1) Unit testing, and a better alternative to assertAlmostEqual. (2) Approximate equality comparisons, as per Guido's example. Note that those two are slightly different: in the unit testing case, you usually have an known expected value (not necessarily mathematically exact, but at least known) while in Guido's example neither value is necessarily better than they other, you just want to stop when they are close enough. Like Nick, I think the first is the more important one. In the second case, anyone writing a numeric algorithm is probably copying an algorithm which already incorporates a fuzzy comparison, or they know enough to write their own. The benefits of a standard solution are convenience and correctness. Assuming unittest provides a well-tested is_close/approx_equal function, why not use it?
Note that in this case, at least, we do want a symmetric version of "is_close", since neither guess nor new_guess is "correct", they are both approximations.
true, but you are also asking the question -- is the new_guess much different that guess. Which points to a asymmetric test -- but either would work.
I can see we're going to have to argue about the "Close To" versus "Close" distinction :-) I suggest that in the interest of not flooding everyone's inboxes, we take that off-list until we have either a concensus or at least agreement that we cannot reach concensus. -- Steve