[Numpy-discussion] assert and allclose

[Ralf Gommers]

On Mon, Apr 18, 2011 at 5:59 AM,  <josef.pktd at gmail.com> wrote:
> Some observations about allclose
>
> I have a test case where I would like to impose absolute
> (logical_)and relative errors
>
> (Pdb) pdf_r
> array([  2.97341655e-90,   3.68756271e-24,   2.01840159e-07,
>         3.98942280e-01,   4.83941449e-01,   1.07981933e-01])
> (Pdb) pdf_st
> array([  3.09721604e-90,   3.69697466e-24,   2.01872148e-07,
>         3.98942181e-01,   4.83941207e-01,   1.07982122e-01])
> (Pdb) pdf_st - pdf_r
> array([  1.23799491e-91,   9.41194691e-27,   3.19891525e-11,
>        -9.95821123e-08,  -2.41764579e-07,   1.89015140e-07])
> (Pdb) (pdf_st - pdf_r) / pdf_r
> array([  4.16354347e-02,   2.55234897e-03,   1.58487551e-04,
>        -2.49615338e-07,  -4.99574028e-07,   1.75043301e-06])
>
> allclose just uses the sum of abolute and relative errors, which I
> think is binding in the wrong way, for what I want.
>
> (Pdb) np.allclose(pdf_st, pdf_r, rtol=1e-4, atol=1e-8)
> True
>
> the relative and absolute errors are to tight on one side and too
> loose on the other side
>
> (Pdb) np.allclose(pdf_st, pdf_r, rtol=1e-2, atol=0)
> False
> (Pdb) np.allclose(pdf_st, pdf_r, rtol=0, atol=1e-8)
> False
>
> tighter bounds for both relative and absolute errors if I impose both separately
>
> (Pdb) np.allclose(pdf_st, pdf_r, rtol=0, atol=1e-6)
> True
> (Pdb) np.allclose(pdf_st, pdf_r, rtol=1e-1, atol=0)
> True
>
> Are there any alternatives to this? (or am I too tired)

This indeed doesn't seem to work well for your purpose, and I can't
think of anything better than doing the logical_and by hand. Perhaps
we can build this behavior (separate atol and rtol comparisons, then
"and" those) into allclose with a keyword to select it?

It would also be cleaner to do the rtol comparison with 2 * abs(a - b)
/ (abs(a) + abs(b), to remove the asymmetric response to input
arguments.

Ralf

> (test cases are for statistical distributions, where all values are
> smaller than one, and some values can be very small)