I have a PR up for review that adds "maximum difference" to the error messages produced by NumPy's testing functions for comparing arrays:

https://github.com/numpy/numpy/pull/12591

Because this changes NumPy's public interface, I'm running it by the mailing list to see if there are any objections or suggestions for improvement.

Example behavior:

>>> x = np.array([1, 2, 3])

>>> y = np.array([1, 2, 3.0001])

>>> np.testing.assert_allclose(x, y)

AssertionError:

Not equal to tolerance rtol=1e-07, atol=0

(mismatch 33.333333333333336%, maximum difference 0.00010000000000021103)

x: array([1, 2, 3])

y: array([1. , 2. , 3.0001])

Motivation: when writing numerical algorithms, I frequently find myself experimenting to pick the right value of atol and rtol for np.testing.assert_allclose(). If I make the tolerance too generous, I risk missing regressions in accuracy, so I usually try to pick the smallest values for which tests pass. This change immediately reveals appropriate values to use for these parameters, so I don't need to guess and check.

One alternative would be print both "atol" and "rtol" numbers directly that could be immediately used in assert_allclose(). Here we effectively only have "atol". The main reason why I didn't do this is that I felt it would add more clutter (rtol is slightly redundant with the displayed array values). But we could probably do this if we're willing to split the error message onto a few more lines, e.g.,

AssertionError:

Not equal to tolerance rtol=1e-07, atol=0

(mismatch 33.333333333333336%,

maximum absolute difference 0.00010000000000021103,

maximum relative difference 3.333333333340368e-05)

x: array([1, 2, 3])

y: array([1. , 2. , 3.0001])

Best,

Stephan