[Numpy-discussion] Mysterious test_pareto failure on Travis

[Charles R Harris]

On Tue, Sep 4, 2012 at 2:37 PM, Nathaniel Smith <njs at pobox.com> wrote:

> On Tue, Sep 4, 2012 at 9:17 PM, Ondřej Čertík <ondrej.certik at gmail.com>
> wrote:
> > On Tue, Sep 4, 2012 at 12:41 PM, Ondřej Čertík <ondrej.certik at gmail.com>
> wrote:
> >> On Tue, Sep 4, 2012 at 12:31 PM, Ondřej Čertík <ondrej.certik at gmail.com>
> wrote:
> >>> On Tue, Sep 4, 2012 at 3:15 AM, Nathaniel Smith <njs at pobox.com> wrote:
> >>>> The last two Travis builds of master have failed consistently with the
> >>>> same error:
> >>>>   http://travis-ci.org/#!/numpy/numpy/builds
> >>>> It looks like a real failure -- we're getting the same error on every
> >>>> build variant, some sort of problem in test_pareto. Example:
> >>>>   http://travis-ci.org/#!/numpy/numpy/jobs/2328823
> >>>>
> >>>> The obvious culprit would be the previous commit, which regenerated
> >>>> mtrand.c with Cython 0.17:
> >>>>
> http://github.com/numpy/numpy/commit/cd9092aa71d23359b33e89d938c55fb14b9bf606
> >>>>
> >>>> What's weird, though, is that that commit passed just fine on Travis:
> >>>>   http://travis-ci.org/#!/numpy/numpy/builds/2313124
> >>>>
> >>>> It's just the two commits since then that failed. But these commits
> >>>> have been 1-line docstring changes, so I don't see how they could have
> >>>> possibly created the problem.
> >>>>
> >>>> Also, the test passes fine with python 2.7 on my laptop with current
> master.
> >>>>
> >>>> Can anyone reproduce this failure? Any ideas what might be going on?
> >>>
> >>> I made this:
> >>>
> >>> https://github.com/numpy/numpy/issues/424
> >>>
> >>> It was me who updated the Cython file. It seemed to be working. I've
> >>> added the issue
> >>> to the release TODO.
> >>
> >> Ok, here is how to reproduce the problem:
> >>
> >> 1) install my numpy-vendor vagrant image (32 bit Ubuntu), as directed
> >> in the README:
> >>
> >> https://github.com/certik/numpy-vendor
> >>
> >> 2) run tests, you'll get:
> >>
> >> https://gist.github.com/3625509
> >
> > So the problem was actually introduced much earlier. Most probably it
> > has never worked
> > in 32bit Ubuntu 12.04. I tried for example this old commit:
> >
> >
> https://github.com/numpy/numpy/commit/3882d65c42acf6d5fff8cc9b3f410bb3e49c8af8
> >
> > and it still fails:
> [...]
> > But in any case, this seems to be a problem with the actual 32bit
> > Ubuntu 12.04 itself. So maybe something in gcc
> > has changed that now triggers the problem.
>
> To be clear, the mismatching value is:
>
> >
> /home/njs/numpy/.tox/py27/local/lib/python2.7/site-packages/numpy/random/tests/test_random.py(363)test_pareto()
> -> np.testing.assert_array_almost_equal(actual, desired, decimal=15)
> (Pdb) actual[1, 0]
> 52828779.702948704
> (Pdb) desired[1, 0]
> 52828779.702948518
>
> So they do match to 14 decimal points, and it's entirely possible that
> this is just a problem of the test being too stringent in requiring 15
> decimal points of match. Maybe the 32-bit GCC is spilling registers
> differently, tripping the famous x86 idiosyncrasy where register
> spilling triggers rounding. But I'd feel better if someone familiar
> with the pareto code could confirm.
>
> I expect it is the tendency of 32 bit gcc to mix 8087 and sse
instructions, which have different floating point register precisions. As a
result the floating point values can depend on the compiler version and
optimization flags. Here it seems that the test is requiring a bit too much
precision, although it would be nice to know what the expected precision of
the algorithm is.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20120904/9721fa48/attachment.html>