[SciPy-Dev] Scipy test-failures and absolute tolerance-levels

[Charles R Harris]

On Mon, Oct 12, 2015 at 9:35 AM, Jamuar, Rohit <rohit.jamuar at intel.com>
wrote:

> Hi,
>
>
>
> I’m one of the engineers from Python Scripting team at Intel Corporation.
> One of the things that we’ve been trying to achieve is to improve
> performance of numerical and scientific computation packages – numpy and
> scipy, for starters. While building Scipy(v0.16) with Intel Compiler, we
> see failures that are being caused by stringent absolute tolerance-levels.
> It also seems that you ran into such problems with these tests earlier
> <https://github.com/scipy/scipy/pull/4243>. These are some of the failing
> tests :
>
> 1.      test_j_roots() (from test_orthogonal in scipy.special) -
>
> a.       vgq(rf(0.5, -0.5), ef(0.5, -0.5), wf(0.5, -0.5), -1., 1., 25,
> atol=1e-13) : If the tolerance is changed to 1e-12, the test passes
>
> 2.      test_js_roots() (from test_orthogonal in scipy.special) -
>
> a.       vgq(rf(1, 0.5), ef(1, 0.5), wf(1, 0.5), 0., 1., 25, atol=1e-13)
> : ) : If the tolerance is changed to 1e-12, the test passes
>
> b.      vgq(rf(68.9, 2.25), ef(68.9, 2.25), wf(68.9, 2.25), 0., 1., 5,
> atol=2e-14) ) : If the tolerance is changed to 2e-13, the test passes
>
>
>
> I would really appreciate if you could share your rationale(s) behind
> altering tolerance-levels – does choosing a new tolerance-level depend on
> some theoretically defined range or is it chosen empirically? This
> information would help us better understand the ramifications of changing
> these values.
>

This should be opened as an issue on the github issue tracker. The
tolerances are usually chosen to be reasonable, but not too loose. It might
be worth trying to track down the reason for the discrepancy here, as the
tolerances should be adequate to account for normal roundoff error.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20151012/db8a6d5c/attachment.html>