[Numpy-discussion] suggestion: improve text of failing test

josef.pktd at gmail.com josef.pktd at gmail.com
Thu Feb 5 15:15:14 EST 2015


The assert_allclose text is not precise enough to be helpful to fix a test
failure that cannot be replicated on every machine, and we cannot just
quickly grab --pdb-failures.

By how much do I have to lower the precision to make it pass on this
continuous integration machine?


assert_allclose(he, hefd, rtol=5e-10)
  File "C:\Python27\envs\py3\lib\site-packages\numpy\testing\utils.py",
line 1297, in assert_allclose
    verbose=verbose, header=header)
  File "C:\Python27\envs\py3\lib\site-packages\numpy\testing\utils.py",
line 665, in assert_array_compare
    raise AssertionError(msg)
AssertionError:
Not equal to tolerance rtol=5e-10, atol=0

(mismatch 100.0%)
 x: array([[ -2.965667e+01,  -1.988865e+02,  -2.370194e+00,  -1.003654e+01],
       [ -1.988865e+02,  -1.383377e+03,  -1.592292e+01,  -6.800266e+01],
       [ -2.370194e+00,  -1.592292e+01,  -8.301699e-01,  -8.301699e-01],
       [ -1.003654e+01,  -6.800266e+01,  -8.301699e-01,  -3.449885e+00]])
 y: array([[ -2.965667e+01,  -1.988865e+02,  -2.370194e+00,  -1.003654e+01],
       [ -1.988865e+02,  -1.383377e+03,  -1.592292e+01,  -6.800266e+01],
       [ -2.370194e+00,  -1.592292e+01,  -8.301699e-01,  -8.301699e-01],
       [ -1.003654e+01,  -6.800266e+01,  -8.301699e-01,  -3.449885e+00]])


the suggestion is to add rtol and atol to the mismatch summary, so we can
see if it's just a precision issue or something serious

rtol = np.max(np.abs(x / y - 1)
atol = np.max(np.abs(x - y)

(mismatch 100.0%  rtol=xxx  atol=xxx)


(and as aside to the "all close" discussion:
I do set the tolerances very carefully especially if the agreement with
comparison numbers is below 1e-6 or so)


Josef
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20150205/f6eff28d/attachment.html>


More information about the NumPy-Discussion mailing list