On 03/28/2016 04:33 PM, Matthew Brett wrote:

Please do test on your own machines with something like this script [4]: Matthew,

I ran the tests after installing the wheels on my machine running Ubuntu 14.04. Three numpy tests failed with the GFORTRAN_1.4 error you mentioned in post to the wheel-builds list recently. All other tests passed. I can reproduce these failing tests in a Docker container if it is helpful. # python -c 'import numpy; numpy.test("full")' Running unit tests for numpy NumPy version 1.11.0 NumPy relaxed strides checking option: False NumPy is installed in /usr/local/lib/python2.7/dist-packages/numpy Python version 2.7.6 (default, Jun 22 2015, 17:58:13) [GCC 4.8.2] nose version 1.3.7 ... ====================================================================== ERROR: test_kind.TestKind.test_all ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/nose/case.py", line 381, in setUp try_run(self.inst, ('setup', 'setUp')) File "/usr/local/lib/python2.7/dist-packages/nose/util.py", line 471, in try_run return func() File "/usr/local/lib/python2.7/dist-packages/numpy/f2py/tests/util.py", line 358, in setUp module_name=self.module_name) File "/usr/local/lib/python2.7/dist-packages/numpy/f2py/tests/util.py", line 78, in wrapper memo[key] = func(*a, **kw) File "/usr/local/lib/python2.7/dist-packages/numpy/f2py/tests/util.py", line 149, in build_module __import__(module_name) ImportError: /usr/local/lib/python2.7/dist-packages/numpy/core/../.libs/libgfortran.so.3: version `GFORTRAN_1.4' not found (required by /tmp/tmptYznnz/_test_ext_module_5405.so) ====================================================================== ERROR: test_mixed.TestMixed.test_all ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/nose/case.py", line 381, in setUp try_run(self.inst, ('setup', 'setUp')) File "/usr/local/lib/python2.7/dist-packages/nose/util.py", line 471, in try_run return func() File "/usr/local/lib/python2.7/dist-packages/numpy/f2py/tests/util.py", line 358, in setUp module_name=self.module_name) File "/usr/local/lib/python2.7/dist-packages/numpy/f2py/tests/util.py", line 78, in wrapper memo[key] = func(*a, **kw) File "/usr/local/lib/python2.7/dist-packages/numpy/f2py/tests/util.py", line 149, in build_module __import__(module_name) ImportError: /usr/local/lib/python2.7/dist-packages/numpy/core/../.libs/libgfortran.so.3: version `GFORTRAN_1.4' not found (required by /tmp/tmptYznnz/_test_ext_module_5405.so) ====================================================================== ERROR: test_mixed.TestMixed.test_docstring ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/nose/case.py", line 381, in setUp try_run(self.inst, ('setup', 'setUp')) File "/usr/local/lib/python2.7/dist-packages/nose/util.py", line 471, in try_run return func() File "/usr/local/lib/python2.7/dist-packages/numpy/f2py/tests/util.py", line 358, in setUp module_name=self.module_name) File "/usr/local/lib/python2.7/dist-packages/numpy/f2py/tests/util.py", line 84, in wrapper raise ret ImportError: /usr/local/lib/python2.7/dist-packages/numpy/core/../.libs/libgfortran.so.3: version `GFORTRAN_1.4' not found (required by /tmp/tmptYznnz/_test_ext_module_5405.so) ---------------------------------------------------------------------- Ran 6322 tests in 136.678s FAILED (KNOWNFAIL=6, SKIP=11, errors=3) Cheers, - Jonathan Helmus

On Nix/NixOS we've been using OpenBLAS 0.2.14 for some time now because we had some segmentation faults with 0.2.15 and scipy/scikitlearn. I've tested the packages you listed, and more, with OpenBLAS 0.2.17 and encountered no problems. On Wed, Mar 30, 2016 at 1:32 PM, Olivier Grisel wrote:

The problem with the gfortran failures will be tackled by renaming the vendored libgfortran.so library, see:

https://github.com/pypa/auditwheel/issues/24

This is orthogonal to the ATLAS vs OpenBLAS decision though.

-- Olivier _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion

I would like to see OpenBLAS support for numpy on windows. The latest OpenBLAS windows builds numpy support for are on https://bitbucket.org/carlkl/mingw-w64-for-python/downloads now. Scipy wheels should work regardless if numpy was build with MSVC or with mingwpy. It is only mandantory to agree about the BLAS implemenation used. Carl 2016-03-30 23:30 GMT+02:00 Nathaniel Smith :

If OpenBLAS is looking like the easiest to support solution, then no objections here. (If 0.2.17 is genuinely working well, then maybe we want to switch to it on Windows too. I know Xianyi disabled some of the problematic kernels for us -- maybe that's enough. Mostly I just don't want to end up in the situation where we're trying to support a bunch of differently broken builds on different platforms.) On Mar 28, 2016 2:34 PM, "Matthew Brett" wrote:

...

Hi,

Olivier Grisel and I are working on building and testing manylinux wheels for numpy and scipy.

We first thought that we should use ATLAS BLAS, but Olivier found that my build of these could be very slow [1]. I set up a testing grid [2] which found test errors for numpy and scipy using ATLAS wheels.

On the other hand, the same testing grid finds no errors or failures [3] using latest OpenBLAS (0.2.17) and running tests for:

numpy scipy scikit-learn numexpr pandas statsmodels

This is on the travis-ci ubuntu VMs.

Please do test on your own machines with something like this script [4]:

source test_manylinux.sh

We have worried in the past about the reliability of OpenBLAS, but I find these tests reassuring.

Are there any other tests of OpenBLAS that we should run to assure ourselves that it is safe to use?

Matthew

[1] https://github.com/matthew-brett/manylinux-builds/issues/4#issue-143530908 [2] https://travis-ci.org/matthew-brett/manylinux-testing/builds/118780781 [3] I disabled a few pandas tests which were failing for reasons not related to BLAS. Some of the statsmodels test runs time out. [4] https://gist.github.com/matthew-brett/2fd9d9a29e022c297634 _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion

On Mon, Mar 28, 2016 at 2:33 PM, Matthew Brett wrote:

Hi,

Olivier Grisel and I are working on building and testing manylinux wheels for numpy and scipy.

We first thought that we should use ATLAS BLAS, but Olivier found that my build of these could be very slow [1]. I set up a testing grid [2] which found test errors for numpy and scipy using ATLAS wheels.

On the other hand, the same testing grid finds no errors or failures [3] using latest OpenBLAS (0.2.17) and running tests for:

numpy scipy scikit-learn numexpr pandas statsmodels

This is on the travis-ci ubuntu VMs.

Please do test on your own machines with something like this script [4]:

source test_manylinux.sh

We have worried in the past about the reliability of OpenBLAS, but I find these tests reassuring.

Are there any other tests of OpenBLAS that we should run to assure ourselves that it is safe to use?

Here is an update on progress: We've now done a lot of testing on the Linux OpenBLAS wheels. They pass all tests on Linux, with Intel kernels: https://travis-ci.org/matthew-brett/manylinux-testing/builds/120825485 http://nipy.bic.berkeley.edu/builders/manylinux-2.7-debian/builds/22 http://nipy.bic.berkeley.edu/builders/manylinux-2.7-fedora/builds/10 Xianyi, the maintainer of OpenBLAS, is very helpfully running the OpenBLAS buildbot nightly tests with numpy and scipy: http://build.openblas.net/builders There is still one BLAS-related failure on these tests on AMD chips: https://github.com/xianyi/OpenBLAS-CI/issues/10 I propose to hold off distributing the OpenBLAS wheels until the OpenBLAS tests are clean on the OpenBLAS buildbots - any objections? Cheers, Matthew

On Apr 5, 2016 10:23 AM, "Matthew Brett" wrote:

On Mon, Mar 28, 2016 at 2:33 PM, Matthew Brett

wrote:

...

Hi,

Olivier Grisel and I are working on building and testing manylinux wheels for numpy and scipy.

We first thought that we should use ATLAS BLAS, but Olivier found that my build of these could be very slow [1]. I set up a testing grid [2] which found test errors for numpy and scipy using ATLAS wheels.

On the other hand, the same testing grid finds no errors or failures [3] using latest OpenBLAS (0.2.17) and running tests for:

numpy scipy scikit-learn numexpr pandas statsmodels

This is on the travis-ci ubuntu VMs.

Please do test on your own machines with something like this script [4]:

source test_manylinux.sh

We have worried in the past about the reliability of OpenBLAS, but I find these tests reassuring.

Are there any other tests of OpenBLAS that we should run to assure ourselves that it is safe to use?

Here is an update on progress:

We've now done a lot of testing on the Linux OpenBLAS wheels. They pass all tests on Linux, with Intel kernels:

https://travis-ci.org/matthew-brett/manylinux-testing/builds/120825485 http://nipy.bic.berkeley.edu/builders/manylinux-2.7-debian/builds/22 http://nipy.bic.berkeley.edu/builders/manylinux-2.7-fedora/builds/10

Xianyi, the maintainer of OpenBLAS, is very helpfully running the OpenBLAS buildbot nightly tests with numpy and scipy:

http://build.openblas.net/builders

There is still one BLAS-related failure on these tests on AMD chips:

https://github.com/xianyi/OpenBLAS-CI/issues/10

I propose to hold off distributing the OpenBLAS wheels until the OpenBLAS tests are clean on the OpenBLAS buildbots - any objections?

Alternatively, would it make sense to add a local patch to our openblas builds to blacklist the piledriver kernel and then distribute them now? (I'm not immediately sure what would be involved in doing this but it seems unlikely that it would require anything tricky?) -n

On Wed, Apr 6, 2016 at 2:04 AM, Olivier Grisel wrote:

2016-04-05 19:44 GMT+02:00 Nathaniel Smith :

...

...

I propose to hold off distributing the OpenBLAS wheels until the OpenBLAS tests are clean on the OpenBLAS buildbots - any objections?

Alternatively, would it make sense to add a local patch to our openblas builds to blacklist the piledriver kernel and then distribute them now? (I'm not immediately sure what would be involved in doing this but it seems unlikely that it would require anything tricky?)

I tried to force use the NEHALEM or the BARCELONA driver on a PILEDRIVER AMD box and while it fixes the original test failure in isolve it causes this other scipy test to fail:

====================================================================== FAIL: test_nanmedian_all_axis (test_stats.TestNanFunc) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/scipy/stats/tests/test_stats.py", line 242, in test_nanmedian_all_axis assert_equal(len(w), 4) File "/usr/local/lib/python2.7/site-packages/numpy/testing/utils.py", line 375, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 1 DESIRED: 4

I'm reading this email next to https://github.com/xianyi/OpenBLAS-CI/issues/10#issuecomment-206195714 and I'm confused :-). -n -- Nathaniel J. Smith -- https://vorpus.org

I updated the issue: https://github.com/xianyi/OpenBLAS-CI/issues/10#issuecomment-206195714 The random test_nanmedian_all_axis failure is unrelated to openblas and should be ignored. -- Olivier