On Thu, Jul 17, 2014 at 4:07 PM, <josef.pktd@gmail.com> wrote:



On Wed, Jul 16, 2014 at 9:52 AM, Nathaniel Smith <njs@pobox.com> wrote:

On 16 Jul 2014 10:26, "Tony Yu" <tsyu80@gmail.com> wrote:
>
> Is there any reason why the defaults for `allclose` and `assert_allclose` differ? This makes debugging a broken test much more difficult. More importantly, using an absolute tolerance of 0 causes failures for some common cases. For example, if two values are very close to zero, a test will fail:
>
>     np.testing.assert_allclose(0, 1e-14)
>
> Git blame suggests the change was made in the following commit, but I guess that change only reverted to the original behavior.
>
> https://github.com/numpy/numpy/commit/f43223479f917e404e724e6a3df27aa701e6d6bf
>
> It seems like the defaults for  `allclose` and `assert_allclose` should match, and an absolute tolerance of 0 is probably not ideal. I guess this is a pretty big behavioral change, but the current default for `assert_allclose` doesn't seem ideal.

What you say makes sense to me, and loosening the default tolerances won't break any existing tests. (And I'm not too worried about people who were counting on getting 1e-7 instead of 1e-5 or whatever... if it matters that much to you exactly what tolerance you test, you should be setting the tolerance explicitly!) I vote that unless someone comes up with some terrible objection in the next few days then you should submit a PR :-)


If you mean by this to add atol=1e-8 as default, then I'm against it.

At least it will change the meaning of many of our tests in statsmodels.

I'm using rtol to check for correct 1e-15 or 1e-30, which would be completely swamped if you change the default atol=0.
Adding atol=0 to all assert_allclose that currently use only rtol is a lot of work.
I think I almost never use a default rtol, but I often leave atol at the default = 0.

If we have zeros, then I don't think it's too much work to decide whether this should be atol=1e-20, or 1e-8.

copied from http://mail.scipy.org/pipermail/numpy-discussion/2014-July/070639.html
since I didn't get any messages here
This is a compelling use-case, but there are also lots of compelling
usecases that want some non-zero atol (i.e., comparing stuff to 0).
Saying that allclose is for one of those use cases and assert_allclose
is for the other is... not a very felicitious API design, I think. So
we really should do *something*.

Are there really any cases where you want non-zero atol= that don't
involve comparing something against a 'desired' value of zero? It's a
little wacky, but I'm wondering if we ought to change the rule (for
all versions of allclose) to

if desired == 0:
    tol = atol
else:
    tol = rtol * desired

In particular, means that np.allclose(x, 1e-30) would reject x values
of 0 or 2e-30, but np.allclose(x, 0) will accept x == 1e-30 or 2e-30.

-n

That's much too confusing.
I don't know what the usecases for np.allclose are since I don't have any.

assert_allclose is one of our (statsmodels) most frequently used numpy function

this is not informative:
`np.allclose(x, 1e-30)`

since there are keywords
either np.assert_allclose(x, atol=1e-30)
if I want to be "close" to zero
or 
np.assert_allclose(x, rtol=1e-11, atol=1e-25)
if we have a mix of large numbers and "zeros" in an array.

Making the behavior of assert_allclose depending on whether desired is exactly zero or 1e-20 looks too difficult to remember, and which desired I use would depend on what I get out of R or Stata.

atol=1e-8 is not close to zero in most cases in my experience.


The numpy.testing assert functions are some of the most useful functions in numpy, and heavily used "code".

Josef

 

Josef

 

-n


_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion