[Numpy-discussion] `allclose` vs `assert_allclose`

[Nathaniel Smith]

On Fri, Jul 18, 2014 at 7:03 PM,  <josef.pktd at gmail.com> wrote:
>
> On Fri, Jul 18, 2014 at 12:53 PM, Nathaniel Smith <njs at pobox.com> wrote:
>>
>> For cases like this, you need *some* non-zero atol or the thing just
>> doesn't work, and one could quibble over the exact value as long as
>> it's larger than "normal" floating point error. These calculations
>> usually involve "normal" sized numbers, so atol should be comparable
>> to eps * these values.  eps is 2e-16, so atol=1e-8 works for values up
>> to around 1e8, which is a plausible upper bound for where people might
>> expect assert_allclose to just work. I'm trying to figure out some way
>> to support your use cases while also supporting other use cases.
>
>
> my problem is that there is no "normal" floating point error.
> If I have units in 1000 or units in 0.0001 depends on the example and
> dataset that we use for testing.
>
> this test two different functions/methods that calculate the same thing
>
> (Pdb) pval
> array([  3.01270184e-42,   5.90847367e-02,   3.00066946e-12])
> (Pdb) res2.pvalues
> array([  3.01270184e-42,   5.90847367e-02,   3.00066946e-12])
> (Pdb) assert_allclose(pval, res2.pvalues, rtol=5 * rtol, atol=1e-25)
>
> I don't care about errors that are smaller that 1e-25
>
> for example testing p-values against Stata
>
> (Pdb) tt.pvalue
> array([  5.70315140e-30,   6.24662551e-02,   5.86024090e-11])
> (Pdb) res2.pvalues
> array([  5.70315140e-30,   6.24662551e-02,   5.86024090e-11])
> (Pdb) tt.pvalue - res2.pvalues
> array([  2.16612016e-40,   2.51187959e-15,   4.30027936e-21])
> (Pdb) tt.pvalue / res2.pvalues - 1
> array([  3.79811738e-11,   4.01900735e-14,   7.33806349e-11])
> (Pdb) rtol
> 1e-10
> (Pdb) assert_allclose(tt.pvalue, res2.pvalues, rtol=5 * rtol)
>
>
> I could find a lot more and maybe nicer examples, since I spend quite a bit
> of time fine tuning unit tests.

...these are all cases where there are not exact zeros, so my proposal
would not affect them?

I can see the argument that we shouldn't provide any default rtol/atol
at all because there is no good default, but... I don't think putting
that big of a barrier in front of newbies writing their first tests is
a good idea.

-n

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org