Mysterious test_pareto failure on Travis
The last two Travis builds of master have failed consistently with the same error: http://travis-ci.org/#!/numpy/numpy/builds It looks like a real failure -- we're getting the same error on every build variant, some sort of problem in test_pareto. Example: http://travis-ci.org/#!/numpy/numpy/jobs/2328823 The obvious culprit would be the previous commit, which regenerated mtrand.c with Cython 0.17: http://github.com/numpy/numpy/commit/cd9092aa71d23359b33e89d938c55fb14b9bf60... What's weird, though, is that that commit passed just fine on Travis: http://travis-ci.org/#!/numpy/numpy/builds/2313124 It's just the two commits since then that failed. But these commits have been 1-line docstring changes, so I don't see how they could have possibly created the problem. Also, the test passes fine with python 2.7 on my laptop with current master. Can anyone reproduce this failure? Any ideas what might be going on? -n
Hi, On Tue, Sep 4, 2012 at 11:15 AM, Nathaniel Smith <njs@pobox.com> wrote:
The last two Travis builds of master have failed consistently with the same error: http://travis-ci.org/#!/numpy/numpy/builds It looks like a real failure -- we're getting the same error on every build variant, some sort of problem in test_pareto. Example: http://travis-ci.org/#!/numpy/numpy/jobs/2328823
The obvious culprit would be the previous commit, which regenerated mtrand.c with Cython 0.17: http://github.com/numpy/numpy/commit/cd9092aa71d23359b33e89d938c55fb14b9bf60...
What's weird, though, is that that commit passed just fine on Travis: http://travis-ci.org/#!/numpy/numpy/builds/2313124
It's just the two commits since then that failed. But these commits have been 1-line docstring changes, so I don't see how they could have possibly created the problem.
Also, the test passes fine with python 2.7 on my laptop with current master.
Can anyone reproduce this failure? Any ideas what might be going on?
I believe Travis just (a couple of days ago?) switched to Ubuntu 12.04 images - could that be the problem? See you, Matthew
On 4 September 2012 12:23, Matthew Brett <matthew.brett@gmail.com> wrote:
On Tue, Sep 4, 2012 at 11:15 AM, Nathaniel Smith <njs@pobox.com> wrote:
The last two Travis builds of master have failed consistently with the same error: http://travis-ci.org/#!/numpy/numpy/builds It looks like a real failure -- we're getting the same error on every build variant, some sort of problem in test_pareto. Example: http://travis-ci.org/#!/numpy/numpy/jobs/2328823
The obvious culprit would be the previous commit, which regenerated mtrand.c with Cython 0.17: http://github.com/numpy/numpy/commit/cd9092aa71d23359b33e89d938c55fb14b9bf60...
What's weird, though, is that that commit passed just fine on Travis: http://travis-ci.org/#!/numpy/numpy/builds/2313124
It's just the two commits since then that failed. But these commits have been 1-line docstring changes, so I don't see how they could have possibly created the problem.
Also, the test passes fine with python 2.7 on my laptop with current master.
Can anyone reproduce this failure? Any ideas what might be going on?
I believe Travis just (a couple of days ago?) switched to Ubuntu 12.04 images - could that be the problem?
For whatever it's worth, tox reports: py24: commands succeeded py25: commands succeeded py26: commands succeeded py27: commands succeeded ERROR: py31: InterpreterNotFound: python3.1 py32: commands succeeded py27-separate: commands succeeded py32-separate: commands succeeded with git revision a72ce7e on my Ubuntu 12.04 machine (64-bit). Cheers, S
On Tue, Sep 4, 2012 at 1:47 PM, Scott Sinclair <scott.sinclair.za@gmail.com> wrote:
On 4 September 2012 12:23, Matthew Brett <matthew.brett@gmail.com> wrote:
On Tue, Sep 4, 2012 at 11:15 AM, Nathaniel Smith <njs@pobox.com> wrote:
The last two Travis builds of master have failed consistently with the same error: http://travis-ci.org/#!/numpy/numpy/builds It looks like a real failure -- we're getting the same error on every build variant, some sort of problem in test_pareto. Example: http://travis-ci.org/#!/numpy/numpy/jobs/2328823
The obvious culprit would be the previous commit, which regenerated mtrand.c with Cython 0.17: http://github.com/numpy/numpy/commit/cd9092aa71d23359b33e89d938c55fb14b9bf60...
What's weird, though, is that that commit passed just fine on Travis: http://travis-ci.org/#!/numpy/numpy/builds/2313124
It's just the two commits since then that failed. But these commits have been 1-line docstring changes, so I don't see how they could have possibly created the problem.
Also, the test passes fine with python 2.7 on my laptop with current master.
Can anyone reproduce this failure? Any ideas what might be going on?
I believe Travis just (a couple of days ago?) switched to Ubuntu 12.04 images - could that be the problem?
For whatever it's worth, tox reports:
py24: commands succeeded py25: commands succeeded py26: commands succeeded py27: commands succeeded ERROR: py31: InterpreterNotFound: python3.1 py32: commands succeeded py27-separate: commands succeeded py32-separate: commands succeeded
with git revision a72ce7e on my Ubuntu 12.04 machine (64-bit).
It works for me on 64-bit 12.04 too. However, the Travis machines are using *32*-bit 12.04, which might be the problem... anyone got one of those lying around to test? -n
Hi, On Tue, Sep 4, 2012 at 2:18 PM, Nathaniel Smith <njs@pobox.com> wrote:
On Tue, Sep 4, 2012 at 1:47 PM, Scott Sinclair <scott.sinclair.za@gmail.com> wrote:
On 4 September 2012 12:23, Matthew Brett <matthew.brett@gmail.com> wrote:
On Tue, Sep 4, 2012 at 11:15 AM, Nathaniel Smith <njs@pobox.com> wrote:
The last two Travis builds of master have failed consistently with the same error: http://travis-ci.org/#!/numpy/numpy/builds It looks like a real failure -- we're getting the same error on every build variant, some sort of problem in test_pareto. Example: http://travis-ci.org/#!/numpy/numpy/jobs/2328823
The obvious culprit would be the previous commit, which regenerated mtrand.c with Cython 0.17: http://github.com/numpy/numpy/commit/cd9092aa71d23359b33e89d938c55fb14b9bf60...
What's weird, though, is that that commit passed just fine on Travis: http://travis-ci.org/#!/numpy/numpy/builds/2313124
It's just the two commits since then that failed. But these commits have been 1-line docstring changes, so I don't see how they could have possibly created the problem.
Also, the test passes fine with python 2.7 on my laptop with current master.
Can anyone reproduce this failure? Any ideas what might be going on?
I believe Travis just (a couple of days ago?) switched to Ubuntu 12.04 images - could that be the problem?
For whatever it's worth, tox reports:
py24: commands succeeded py25: commands succeeded py26: commands succeeded py27: commands succeeded ERROR: py31: InterpreterNotFound: python3.1 py32: commands succeeded py27-separate: commands succeeded py32-separate: commands succeeded
with git revision a72ce7e on my Ubuntu 12.04 machine (64-bit).
It works for me on 64-bit 12.04 too. However, the Travis machines are using *32*-bit 12.04, which might be the problem... anyone got one of those lying around to test?
I do - send me your ssh public key somehow? See you, Matthew
On Tue, Sep 4, 2012 at 3:15 AM, Nathaniel Smith <njs@pobox.com> wrote:
The last two Travis builds of master have failed consistently with the same error: http://travis-ci.org/#!/numpy/numpy/builds It looks like a real failure -- we're getting the same error on every build variant, some sort of problem in test_pareto. Example: http://travis-ci.org/#!/numpy/numpy/jobs/2328823
The obvious culprit would be the previous commit, which regenerated mtrand.c with Cython 0.17: http://github.com/numpy/numpy/commit/cd9092aa71d23359b33e89d938c55fb14b9bf60...
What's weird, though, is that that commit passed just fine on Travis: http://travis-ci.org/#!/numpy/numpy/builds/2313124
It's just the two commits since then that failed. But these commits have been 1-line docstring changes, so I don't see how they could have possibly created the problem.
Also, the test passes fine with python 2.7 on my laptop with current master.
Can anyone reproduce this failure? Any ideas what might be going on?
I made this: https://github.com/numpy/numpy/issues/424 It was me who updated the Cython file. It seemed to be working. I've added the issue to the release TODO. Ondrej
On Tue, Sep 4, 2012 at 12:31 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Tue, Sep 4, 2012 at 3:15 AM, Nathaniel Smith <njs@pobox.com> wrote:
The last two Travis builds of master have failed consistently with the same error: http://travis-ci.org/#!/numpy/numpy/builds It looks like a real failure -- we're getting the same error on every build variant, some sort of problem in test_pareto. Example: http://travis-ci.org/#!/numpy/numpy/jobs/2328823
The obvious culprit would be the previous commit, which regenerated mtrand.c with Cython 0.17: http://github.com/numpy/numpy/commit/cd9092aa71d23359b33e89d938c55fb14b9bf60...
What's weird, though, is that that commit passed just fine on Travis: http://travis-ci.org/#!/numpy/numpy/builds/2313124
It's just the two commits since then that failed. But these commits have been 1-line docstring changes, so I don't see how they could have possibly created the problem.
Also, the test passes fine with python 2.7 on my laptop with current master.
Can anyone reproduce this failure? Any ideas what might be going on?
I made this:
https://github.com/numpy/numpy/issues/424
It was me who updated the Cython file. It seemed to be working. I've added the issue to the release TODO.
Ok, here is how to reproduce the problem: 1) install my numpy-vendor vagrant image (32 bit Ubuntu), as directed in the README: https://github.com/certik/numpy-vendor 2) run tests, you'll get: https://gist.github.com/3625509 Ondrej
On Tue, Sep 4, 2012 at 12:41 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Tue, Sep 4, 2012 at 12:31 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Tue, Sep 4, 2012 at 3:15 AM, Nathaniel Smith <njs@pobox.com> wrote:
The last two Travis builds of master have failed consistently with the same error: http://travis-ci.org/#!/numpy/numpy/builds It looks like a real failure -- we're getting the same error on every build variant, some sort of problem in test_pareto. Example: http://travis-ci.org/#!/numpy/numpy/jobs/2328823
The obvious culprit would be the previous commit, which regenerated mtrand.c with Cython 0.17: http://github.com/numpy/numpy/commit/cd9092aa71d23359b33e89d938c55fb14b9bf60...
What's weird, though, is that that commit passed just fine on Travis: http://travis-ci.org/#!/numpy/numpy/builds/2313124
It's just the two commits since then that failed. But these commits have been 1-line docstring changes, so I don't see how they could have possibly created the problem.
Also, the test passes fine with python 2.7 on my laptop with current master.
Can anyone reproduce this failure? Any ideas what might be going on?
I made this:
https://github.com/numpy/numpy/issues/424
It was me who updated the Cython file. It seemed to be working. I've added the issue to the release TODO.
Ok, here is how to reproduce the problem:
1) install my numpy-vendor vagrant image (32 bit Ubuntu), as directed in the README:
https://github.com/certik/numpy-vendor
2) run tests, you'll get:
So the problem was actually introduced much earlier. Most probably it has never worked in 32bit Ubuntu 12.04. I tried for example this old commit: https://github.com/numpy/numpy/commit/3882d65c42acf6d5fff8cc9b3f410bb3e49c8a... and it still fails: https://gist.github.com/3625943 I tried to test even the first commit that introduced the problem: https://github.com/numpy/numpy/commit/898e6bdc625cdd3c97865ef99f8d51c5f43eaf... but while it compiled, I got some import error: (py)vagrant@precise32:~/repos/numpy/tools$ nosetests /home/vagrant/repos/numpy/py/local/lib/python2.7/site-packages/numpy/random/tests/test_random.py RuntimeError: module compiled against API version 6 but this version of numpy is 5 E ====================================================================== ERROR: Failure: ImportError (numpy.core.multiarray failed to import) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/vagrant/repos/numpy/py/local/lib/python2.7/site-packages/nose/loader.py", line 390, in loadTestsFromName addr.filename, addr.module) File "/home/vagrant/repos/numpy/py/local/lib/python2.7/site-packages/nose/importer.py", line 39, in importFromPath return self.importFromDir(dir_path, fqname) File "/home/vagrant/repos/numpy/py/local/lib/python2.7/site-packages/nose/importer.py", line 86, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "/home/vagrant/repos/numpy/py/local/lib/python2.7/site-packages/numpy/random/tests/test_random.py", line 1, in <module> from numpy.testing import TestCase, run_module_suite, assert_ File "/home/vagrant/repos/numpy/py/local/lib/python2.7/site-packages/numpy/__init__.py", line 137, in <module> import add_newdocs File "/home/vagrant/repos/numpy/py/local/lib/python2.7/site-packages/numpy/add_newdocs.py", line 9, in <module> from numpy.lib import add_newdoc File "/home/vagrant/repos/numpy/py/local/lib/python2.7/site-packages/numpy/lib/__init__.py", line 4, in <module> from type_check import * File "/home/vagrant/repos/numpy/py/local/lib/python2.7/site-packages/numpy/lib/type_check.py", line 8, in <module> import numpy.core.numeric as _nx File "/home/vagrant/repos/numpy/py/local/lib/python2.7/site-packages/numpy/core/__init__.py", line 10, in <module> import _sort ImportError: numpy.core.multiarray failed to import ---------------------------------------------------------------------- Ran 1 test in 0.002s FAILED (errors=1) But in any case, this seems to be a problem with the actual 32bit Ubuntu 12.04 itself. So maybe something in gcc has changed that now triggers the problem. Ondrej
On Tue, Sep 4, 2012 at 9:17 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Tue, Sep 4, 2012 at 12:41 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Tue, Sep 4, 2012 at 12:31 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Tue, Sep 4, 2012 at 3:15 AM, Nathaniel Smith <njs@pobox.com> wrote:
The last two Travis builds of master have failed consistently with the same error: http://travis-ci.org/#!/numpy/numpy/builds It looks like a real failure -- we're getting the same error on every build variant, some sort of problem in test_pareto. Example: http://travis-ci.org/#!/numpy/numpy/jobs/2328823
The obvious culprit would be the previous commit, which regenerated mtrand.c with Cython 0.17: http://github.com/numpy/numpy/commit/cd9092aa71d23359b33e89d938c55fb14b9bf60...
What's weird, though, is that that commit passed just fine on Travis: http://travis-ci.org/#!/numpy/numpy/builds/2313124
It's just the two commits since then that failed. But these commits have been 1-line docstring changes, so I don't see how they could have possibly created the problem.
Also, the test passes fine with python 2.7 on my laptop with current master.
Can anyone reproduce this failure? Any ideas what might be going on?
I made this:
https://github.com/numpy/numpy/issues/424
It was me who updated the Cython file. It seemed to be working. I've added the issue to the release TODO.
Ok, here is how to reproduce the problem:
1) install my numpy-vendor vagrant image (32 bit Ubuntu), as directed in the README:
https://github.com/certik/numpy-vendor
2) run tests, you'll get:
So the problem was actually introduced much earlier. Most probably it has never worked in 32bit Ubuntu 12.04. I tried for example this old commit:
https://github.com/numpy/numpy/commit/3882d65c42acf6d5fff8cc9b3f410bb3e49c8a...
and it still fails: [...] But in any case, this seems to be a problem with the actual 32bit Ubuntu 12.04 itself. So maybe something in gcc has changed that now triggers the problem.
To be clear, the mismatching value is:
/home/njs/numpy/.tox/py27/local/lib/python2.7/site-packages/numpy/random/tests/test_random.py(363)test_pareto() -> np.testing.assert_array_almost_equal(actual, desired, decimal=15) (Pdb) actual[1, 0] 52828779.702948704 (Pdb) desired[1, 0] 52828779.702948518
So they do match to 14 decimal points, and it's entirely possible that this is just a problem of the test being too stringent in requiring 15 decimal points of match. Maybe the 32-bit GCC is spilling registers differently, tripping the famous x86 idiosyncrasy where register spilling triggers rounding. But I'd feel better if someone familiar with the pareto code could confirm. -n
On Tue, Sep 4, 2012 at 4:37 PM, Nathaniel Smith <njs@pobox.com> wrote:
On Tue, Sep 4, 2012 at 9:17 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Tue, Sep 4, 2012 at 12:41 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Tue, Sep 4, 2012 at 12:31 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Tue, Sep 4, 2012 at 3:15 AM, Nathaniel Smith <njs@pobox.com> wrote:
The last two Travis builds of master have failed consistently with the same error: http://travis-ci.org/#!/numpy/numpy/builds It looks like a real failure -- we're getting the same error on every build variant, some sort of problem in test_pareto. Example: http://travis-ci.org/#!/numpy/numpy/jobs/2328823
The obvious culprit would be the previous commit, which regenerated mtrand.c with Cython 0.17: http://github.com/numpy/numpy/commit/cd9092aa71d23359b33e89d938c55fb14b9bf60...
What's weird, though, is that that commit passed just fine on Travis: http://travis-ci.org/#!/numpy/numpy/builds/2313124
It's just the two commits since then that failed. But these commits have been 1-line docstring changes, so I don't see how they could have possibly created the problem.
Also, the test passes fine with python 2.7 on my laptop with current master.
Can anyone reproduce this failure? Any ideas what might be going on?
I made this:
https://github.com/numpy/numpy/issues/424
It was me who updated the Cython file. It seemed to be working. I've added the issue to the release TODO.
Ok, here is how to reproduce the problem:
1) install my numpy-vendor vagrant image (32 bit Ubuntu), as directed in the README:
https://github.com/certik/numpy-vendor
2) run tests, you'll get:
So the problem was actually introduced much earlier. Most probably it has never worked in 32bit Ubuntu 12.04. I tried for example this old commit:
https://github.com/numpy/numpy/commit/3882d65c42acf6d5fff8cc9b3f410bb3e49c8a...
and it still fails: [...] But in any case, this seems to be a problem with the actual 32bit Ubuntu 12.04 itself. So maybe something in gcc has changed that now triggers the problem.
To be clear, the mismatching value is:
/home/njs/numpy/.tox/py27/local/lib/python2.7/site-packages/numpy/random/tests/test_random.py(363)test_pareto() -> np.testing.assert_array_almost_equal(actual, desired, decimal=15) (Pdb) actual[1, 0] 52828779.702948704 (Pdb) desired[1, 0] 52828779.702948518
So they do match to 14 decimal points, and it's entirely possible that this is just a problem of the test being too stringent in requiring 15 decimal points of match. Maybe the 32-bit GCC is spilling registers differently, tripping the famous x86 idiosyncrasy where register spilling triggers rounding. But I'd feel better if someone familiar with the pareto code could confirm.
I don't understand this. Isn't assert_almost_equal testing decimals not significant digits? As I remember, these tests were added to avoid or signal changes in the random number generator, and I don't think a digit up or down makes a difference. Josef
-n _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Tue, Sep 4, 2012 at 1:48 PM, <josef.pktd@gmail.com> wrote:
On Tue, Sep 4, 2012 at 4:37 PM, Nathaniel Smith <njs@pobox.com> wrote:
On Tue, Sep 4, 2012 at 9:17 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Tue, Sep 4, 2012 at 12:41 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Tue, Sep 4, 2012 at 12:31 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Tue, Sep 4, 2012 at 3:15 AM, Nathaniel Smith <njs@pobox.com> wrote:
The last two Travis builds of master have failed consistently with the same error: http://travis-ci.org/#!/numpy/numpy/builds It looks like a real failure -- we're getting the same error on every build variant, some sort of problem in test_pareto. Example: http://travis-ci.org/#!/numpy/numpy/jobs/2328823
The obvious culprit would be the previous commit, which regenerated mtrand.c with Cython 0.17: http://github.com/numpy/numpy/commit/cd9092aa71d23359b33e89d938c55fb14b9bf60...
What's weird, though, is that that commit passed just fine on Travis: http://travis-ci.org/#!/numpy/numpy/builds/2313124
It's just the two commits since then that failed. But these commits have been 1-line docstring changes, so I don't see how they could have possibly created the problem.
Also, the test passes fine with python 2.7 on my laptop with current master.
Can anyone reproduce this failure? Any ideas what might be going on?
I made this:
https://github.com/numpy/numpy/issues/424
It was me who updated the Cython file. It seemed to be working. I've added the issue to the release TODO.
Ok, here is how to reproduce the problem:
1) install my numpy-vendor vagrant image (32 bit Ubuntu), as directed in the README:
https://github.com/certik/numpy-vendor
2) run tests, you'll get:
So the problem was actually introduced much earlier. Most probably it has never worked in 32bit Ubuntu 12.04. I tried for example this old commit:
https://github.com/numpy/numpy/commit/3882d65c42acf6d5fff8cc9b3f410bb3e49c8a...
and it still fails: [...] But in any case, this seems to be a problem with the actual 32bit Ubuntu 12.04 itself. So maybe something in gcc has changed that now triggers the problem.
To be clear, the mismatching value is:
/home/njs/numpy/.tox/py27/local/lib/python2.7/site-packages/numpy/random/tests/test_random.py(363)test_pareto() -> np.testing.assert_array_almost_equal(actual, desired, decimal=15) (Pdb) actual[1, 0] 52828779.702948704 (Pdb) desired[1, 0] 52828779.702948518
So they do match to 14 decimal points, and it's entirely possible that this is just a problem of the test being too stringent in requiring 15 decimal points of match. Maybe the 32-bit GCC is spilling registers differently, tripping the famous x86 idiosyncrasy where register spilling triggers rounding. But I'd feel better if someone familiar with the pareto code could confirm.
I don't understand this. Isn't assert_almost_equal testing decimals not significant digits?
Yes it is, see the docs: The test is equivalent to ``abs(desired-actual) < 0.5 * 10**(-decimal)``. and the unit test is badly written, because that floating point number has maybe 14 or 15 good significant digits, but it simply does not have 15 digits after the decimal point, so in particular, that test is testing that the two floating point numbers are exactly equal. Here is a fix: https://github.com/numpy/numpy/pull/425 Let me know if it is ok to merge it, so that we can work on other issues and have a working test suite. Ondrej
On Tue, Sep 4, 2012 at 5:49 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Tue, Sep 4, 2012 at 1:48 PM, <josef.pktd@gmail.com> wrote:
On Tue, Sep 4, 2012 at 4:37 PM, Nathaniel Smith <njs@pobox.com> wrote:
On Tue, Sep 4, 2012 at 9:17 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Tue, Sep 4, 2012 at 12:41 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Tue, Sep 4, 2012 at 12:31 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Tue, Sep 4, 2012 at 3:15 AM, Nathaniel Smith <njs@pobox.com> wrote: > The last two Travis builds of master have failed consistently with the > same error: > http://travis-ci.org/#!/numpy/numpy/builds > It looks like a real failure -- we're getting the same error on every > build variant, some sort of problem in test_pareto. Example: > http://travis-ci.org/#!/numpy/numpy/jobs/2328823 > > The obvious culprit would be the previous commit, which regenerated > mtrand.c with Cython 0.17: > http://github.com/numpy/numpy/commit/cd9092aa71d23359b33e89d938c55fb14b9bf60... > > What's weird, though, is that that commit passed just fine on Travis: > http://travis-ci.org/#!/numpy/numpy/builds/2313124 > > It's just the two commits since then that failed. But these commits > have been 1-line docstring changes, so I don't see how they could have > possibly created the problem. > > Also, the test passes fine with python 2.7 on my laptop with current master. > > Can anyone reproduce this failure? Any ideas what might be going on?
I made this:
https://github.com/numpy/numpy/issues/424
It was me who updated the Cython file. It seemed to be working. I've added the issue to the release TODO.
Ok, here is how to reproduce the problem:
1) install my numpy-vendor vagrant image (32 bit Ubuntu), as directed in the README:
https://github.com/certik/numpy-vendor
2) run tests, you'll get:
So the problem was actually introduced much earlier. Most probably it has never worked in 32bit Ubuntu 12.04. I tried for example this old commit:
https://github.com/numpy/numpy/commit/3882d65c42acf6d5fff8cc9b3f410bb3e49c8a...
and it still fails: [...] But in any case, this seems to be a problem with the actual 32bit Ubuntu 12.04 itself. So maybe something in gcc has changed that now triggers the problem.
To be clear, the mismatching value is:
/home/njs/numpy/.tox/py27/local/lib/python2.7/site-packages/numpy/random/tests/test_random.py(363)test_pareto() -> np.testing.assert_array_almost_equal(actual, desired, decimal=15) (Pdb) actual[1, 0] 52828779.702948704 (Pdb) desired[1, 0] 52828779.702948518
So they do match to 14 decimal points, and it's entirely possible that this is just a problem of the test being too stringent in requiring 15 decimal points of match. Maybe the 32-bit GCC is spilling registers differently, tripping the famous x86 idiosyncrasy where register spilling triggers rounding. But I'd feel better if someone familiar with the pareto code could confirm.
I don't understand this. Isn't assert_almost_equal testing decimals not significant digits?
Yes it is, see the docs:
The test is equivalent to ``abs(desired-actual) < 0.5 * 10**(-decimal)``.
and the unit test is badly written, because that floating point number has maybe 14 or 15 good significant digits, but it simply does not have 15 digits after the decimal point, so in particular, that test is testing that the two floating point numbers are exactly equal.
Here is a fix:
https://github.com/numpy/numpy/pull/425
Let me know if it is ok to merge it, so that we can work on other issues and have a working test suite.
I think all the tests should be changed to use the new assert with relative tolerance (similar to approx equal) https://gist.github.com/3625943 shows also the other distributions with 15 decimals when the values are larger than 10 for example Josef
Ondrej _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Tue, Sep 4, 2012 at 2:58 PM, <josef.pktd@gmail.com> wrote:
On Tue, Sep 4, 2012 at 5:49 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Tue, Sep 4, 2012 at 1:48 PM, <josef.pktd@gmail.com> wrote:
On Tue, Sep 4, 2012 at 4:37 PM, Nathaniel Smith <njs@pobox.com> wrote:
On Tue, Sep 4, 2012 at 9:17 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Tue, Sep 4, 2012 at 12:41 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Tue, Sep 4, 2012 at 12:31 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote: > On Tue, Sep 4, 2012 at 3:15 AM, Nathaniel Smith <njs@pobox.com> wrote: >> The last two Travis builds of master have failed consistently with the >> same error: >> http://travis-ci.org/#!/numpy/numpy/builds >> It looks like a real failure -- we're getting the same error on every >> build variant, some sort of problem in test_pareto. Example: >> http://travis-ci.org/#!/numpy/numpy/jobs/2328823 >> >> The obvious culprit would be the previous commit, which regenerated >> mtrand.c with Cython 0.17: >> http://github.com/numpy/numpy/commit/cd9092aa71d23359b33e89d938c55fb14b9bf60... >> >> What's weird, though, is that that commit passed just fine on Travis: >> http://travis-ci.org/#!/numpy/numpy/builds/2313124 >> >> It's just the two commits since then that failed. But these commits >> have been 1-line docstring changes, so I don't see how they could have >> possibly created the problem. >> >> Also, the test passes fine with python 2.7 on my laptop with current master. >> >> Can anyone reproduce this failure? Any ideas what might be going on? > > I made this: > > https://github.com/numpy/numpy/issues/424 > > It was me who updated the Cython file. It seemed to be working. I've > added the issue > to the release TODO.
Ok, here is how to reproduce the problem:
1) install my numpy-vendor vagrant image (32 bit Ubuntu), as directed in the README:
https://github.com/certik/numpy-vendor
2) run tests, you'll get:
So the problem was actually introduced much earlier. Most probably it has never worked in 32bit Ubuntu 12.04. I tried for example this old commit:
https://github.com/numpy/numpy/commit/3882d65c42acf6d5fff8cc9b3f410bb3e49c8a...
and it still fails: [...] But in any case, this seems to be a problem with the actual 32bit Ubuntu 12.04 itself. So maybe something in gcc has changed that now triggers the problem.
To be clear, the mismatching value is:
/home/njs/numpy/.tox/py27/local/lib/python2.7/site-packages/numpy/random/tests/test_random.py(363)test_pareto() -> np.testing.assert_array_almost_equal(actual, desired, decimal=15) (Pdb) actual[1, 0] 52828779.702948704 (Pdb) desired[1, 0] 52828779.702948518
So they do match to 14 decimal points, and it's entirely possible that this is just a problem of the test being too stringent in requiring 15 decimal points of match. Maybe the 32-bit GCC is spilling registers differently, tripping the famous x86 idiosyncrasy where register spilling triggers rounding. But I'd feel better if someone familiar with the pareto code could confirm.
I don't understand this. Isn't assert_almost_equal testing decimals not significant digits?
Yes it is, see the docs:
The test is equivalent to ``abs(desired-actual) < 0.5 * 10**(-decimal)``.
and the unit test is badly written, because that floating point number has maybe 14 or 15 good significant digits, but it simply does not have 15 digits after the decimal point, so in particular, that test is testing that the two floating point numbers are exactly equal.
Here is a fix:
https://github.com/numpy/numpy/pull/425
Let me know if it is ok to merge it, so that we can work on other issues and have a working test suite.
I think all the tests should be changed to use the new assert with relative tolerance (similar to approx equal) https://gist.github.com/3625943 shows also the other distributions with 15 decimals when the values are larger than 10 for example
That sounds reasonable. Josef, would you mind sending a PR with this better fix? The PR #425 fixes it for now and for me it's now very important to get all the tests pass again on Travis due to their upgrade. I am now working on other failures at Travis: https://github.com/numpy/numpy/issues/394 https://github.com/numpy/numpy/issues/426 As those seem to be causing all the Debian buildbots test failures reported by Sandro. Ondrej
On Tue, Sep 4, 2012 at 9:17 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Tue, Sep 4, 2012 at 12:41 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Tue, Sep 4, 2012 at 12:31 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Tue, Sep 4, 2012 at 3:15 AM, Nathaniel Smith <njs@pobox.com> wrote:
The last two Travis builds of master have failed consistently with the same error: http://travis-ci.org/#!/numpy/numpy/builds It looks like a real failure -- we're getting the same error on every build variant, some sort of problem in test_pareto. Example: http://travis-ci.org/#!/numpy/numpy/jobs/2328823
The obvious culprit would be the previous commit, which regenerated mtrand.c with Cython 0.17:
http://github.com/numpy/numpy/commit/cd9092aa71d23359b33e89d938c55fb14b9bf60...
What's weird, though, is that that commit passed just fine on Travis: http://travis-ci.org/#!/numpy/numpy/builds/2313124
It's just the two commits since then that failed. But these commits have been 1-line docstring changes, so I don't see how they could have possibly created the problem.
Also, the test passes fine with python 2.7 on my laptop with current
master.
Can anyone reproduce this failure? Any ideas what might be going on?
I made this:
https://github.com/numpy/numpy/issues/424
It was me who updated the Cython file. It seemed to be working. I've added the issue to the release TODO.
Ok, here is how to reproduce the problem:
1) install my numpy-vendor vagrant image (32 bit Ubuntu), as directed in the README:
https://github.com/certik/numpy-vendor
2) run tests, you'll get:
So the problem was actually introduced much earlier. Most probably it has never worked in 32bit Ubuntu 12.04. I tried for example this old commit:
https://github.com/numpy/numpy/commit/3882d65c42acf6d5fff8cc9b3f410bb3e49c8a...
and it still fails:
[...]
But in any case, this seems to be a problem with the actual 32bit Ubuntu 12.04 itself. So maybe something in gcc has changed that now triggers the problem.
To be clear, the mismatching value is:
/home/njs/numpy/.tox/py27/local/lib/python2.7/site-packages/numpy/random/tests/test_random.py(363)test_pareto() -> np.testing.assert_array_almost_equal(actual, desired, decimal=15) (Pdb) actual[1, 0] 52828779.702948704 (Pdb) desired[1, 0] 52828779.702948518
So they do match to 14 decimal points, and it's entirely possible that this is just a problem of the test being too stringent in requiring 15 decimal points of match. Maybe the 32-bit GCC is spilling registers differently, tripping the famous x86 idiosyncrasy where register spilling triggers rounding. But I'd feel better if someone familiar with the pareto code could confirm.
I expect it is the tendency of 32 bit gcc to mix 8087 and sse instructions, which have different floating point register precisions. As a result the floating point values can depend on the compiler version and optimization flags. Here it seems that the test is requiring a bit too much
On Tue, Sep 4, 2012 at 2:37 PM, Nathaniel Smith <njs@pobox.com> wrote: precision, although it would be nice to know what the expected precision of the algorithm is. Chuck
participants (6)
-
Charles R Harris -
josef.pktd＠gmail.com -
Matthew Brett -
Nathaniel Smith -
Ondřej Čertík -
Scott Sinclair