On Fri, Jul 18, 2014 at 2:29 PM, Nathaniel Smith <njs@pobox.com> wrote:
On Fri, Jul 18, 2014 at 7:03 PM, <josef.pktd@gmail.com> wrote:
On Fri, Jul 18, 2014 at 12:53 PM, Nathaniel Smith <njs@pobox.com> wrote:
For cases like this, you need *some* non-zero atol or the thing just doesn't work, and one could quibble over the exact value as long as it's larger than "normal" floating point error. These calculations usually involve "normal" sized numbers, so atol should be comparable to eps * these values. eps is 2e-16, so atol=1e-8 works for values up to around 1e8, which is a plausible upper bound for where people might expect assert_allclose to just work. I'm trying to figure out some way to support your use cases while also supporting other use cases.
my problem is that there is no "normal" floating point error. If I have units in 1000 or units in 0.0001 depends on the example and dataset that we use for testing.
this test two different functions/methods that calculate the same thing
(Pdb) pval array([ 3.01270184e-42, 5.90847367e-02, 3.00066946e-12]) (Pdb) res2.pvalues array([ 3.01270184e-42, 5.90847367e-02, 3.00066946e-12]) (Pdb) assert_allclose(pval, res2.pvalues, rtol=5 * rtol, atol=1e-25)
I don't care about errors that are smaller that 1e-25
for example testing p-values against Stata
(Pdb) tt.pvalue array([ 5.70315140e-30, 6.24662551e-02, 5.86024090e-11]) (Pdb) res2.pvalues array([ 5.70315140e-30, 6.24662551e-02, 5.86024090e-11]) (Pdb) tt.pvalue - res2.pvalues array([ 2.16612016e-40, 2.51187959e-15, 4.30027936e-21]) (Pdb) tt.pvalue / res2.pvalues - 1 array([ 3.79811738e-11, 4.01900735e-14, 7.33806349e-11]) (Pdb) rtol 1e-10 (Pdb) assert_allclose(tt.pvalue, res2.pvalues, rtol=5 * rtol)
I could find a lot more and maybe nicer examples, since I spend quite a
bit
of time fine tuning unit tests.
...these are all cases where there are not exact zeros, so my proposal would not affect them?
I can see the argument that we shouldn't provide any default rtol/atol at all because there is no good default, but... I don't think putting that big of a barrier in front of newbies writing their first tests is a good idea.
I think atol=0 is **very** good for newbies, and everyone else. If expected is really zero or very small, then it immediately causes a test failure, and it's relatively obvious how to fix it. I worry a lot more about unit tests that don't "bite" written by newbies or not so newbies who just use a default. That's one of the problems we had with assert_almost_equal, and why I was very happy to switch to assert_allclose with it's emphasis on relative tolerance. Josef
-n
-- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion