[Numpy-discussion] suggestion: improve text of failing test

Nathaniel Smith njs at pobox.com
Thu Feb 5 15:39:27 EST 2015


On 5 Feb 2015 12:15, <josef.pktd at gmail.com> wrote:
>
> The assert_allclose text is not precise enough to be helpful to fix a
test failure that cannot be replicated on every machine, and we cannot just
quickly grab --pdb-failures.
>
> By how much do I have to lower the precision to make it pass on this
continuous integration machine?
>
>
> assert_allclose(he, hefd, rtol=5e-10)
>   File "C:\Python27\envs\py3\lib\site-packages\numpy\testing\utils.py",
line 1297, in assert_allclose
>     verbose=verbose, header=header)
>   File "C:\Python27\envs\py3\lib\site-packages\numpy\testing\utils.py",
line 665, in assert_array_compare
>     raise AssertionError(msg)
> AssertionError:
> Not equal to tolerance rtol=5e-10, atol=0
>
> (mismatch 100.0%)
>  x: array([[ -2.965667e+01,  -1.988865e+02,  -2.370194e+00,
 -1.003654e+01],
>        [ -1.988865e+02,  -1.383377e+03,  -1.592292e+01,  -6.800266e+01],
>        [ -2.370194e+00,  -1.592292e+01,  -8.301699e-01,  -8.301699e-01],
>        [ -1.003654e+01,  -6.800266e+01,  -8.301699e-01,  -3.449885e+00]])
>  y: array([[ -2.965667e+01,  -1.988865e+02,  -2.370194e+00,
 -1.003654e+01],
>        [ -1.988865e+02,  -1.383377e+03,  -1.592292e+01,  -6.800266e+01],
>        [ -2.370194e+00,  -1.592292e+01,  -8.301699e-01,  -8.301699e-01],
>        [ -1.003654e+01,  -6.800266e+01,  -8.301699e-01,  -3.449885e+00]])
>
>
> the suggestion is to add rtol and atol to the mismatch summary, so we can
see if it's just a precision issue or something serious
>
> rtol = np.max(np.abs(x / y - 1)
> atol = np.max(np.abs(x - y)
>
> (mismatch 100.0%  rtol=xxx  atol=xxx)

So basically just printing what rtol and/or atol would have to be to make
the test pass? Sounds useful to me. (There is a bit of an infelicity in
that if you're using both atol and rtol in the same test then there's no
easy way to suggest how to fix both simultaneously, but I'm not sure how to
fix that. Maybe we should also print max(abs(x[y == 0]))?)

Want to submit a pull request?

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20150205/53cc7880/attachment.html>


More information about the NumPy-Discussion mailing list