[Numpy-discussion] Switch to using ATLAS for OSX binary wheels

[Matthew Brett]

On Friday, June 13, 2014, Ralf Gommers <ralf.gommers at gmail.com> wrote:

>
>
>
> On Fri, Jun 13, 2014 at 2:07 PM, Matthew Brett <matthew.brett at gmail.com
> <javascript:_e(%7B%7D,'cvml','matthew.brett at gmail.com');>> wrote:
>
>> Hi,
>>
>> Summary : I'm planning to upload OSX wheels for numpy and scipy using
>> the ATLAS blas / lapack library instead of the default OSX Accelerate
>> framework.
>>
>> We've run into some trouble with a segfault in recent OSX Accelerate:
>>
>> https://github.com/numpy/numpy/issues/4007
>>
>> and Accelerate also doesn't play well with multiprocessing.
>>
>> https://github.com/numpy/numpy/issues/4776
>>
>> Because there's nothing I love more than half-day compilation runs on
>> my laptop, I've built ATLAS binaries with gcc 4.8, and linked numpy
>> and scipy to them to make OSX wheels.  These pass all tests in i386
>> and x86_64 mode, including numpy, scipy, matplotlib, pandas:
>>
>>
>> https://travis-ci.org/matthew-brett/scipy-stack-osx-testing/builds/27442987
>>
>> The build process needs some automating, but it's recorded here:
>>
>> https://github.com/matthew-brett/numpy-atlas-binaries
>>
>> It's possible to get travis-ci to build these guys from a bare machine
>> and then upload them somewhere, but I haven't tried to do that.
>>
>> Meanwhile Sturla kindly worked up a patch to numpy to work round the
>> Accelerate segfault [1].  I haven't tested that, but given I'd already
>> built the wheels, I prefer the ATLAS builds because they work with
>> multiprocessing.
>>
>> I propose uploading these wheels as the default for numpy and scipy.
>> Does anyone have any objection or comments before I go ahead and do
>> that?
>>
>
> From your README and wscript I don't see what numpy version you're using
> to compile scipy against. I got the impression that you used 1.8.1, but it
> should be numpy 1.5.1 for the 2.7 build, and 1.7.1 for 3.x.
>
> I've tried the scipy 0.14.0 python2.7 wheel, but I get import errors (see
> below). Your wheels should work with all common Python installs (mine is
> homebrew) right?
>
> Ralf
>
>
>     $ python2.7 -c "import scipy; scipy.test()"
>     Running unit tests for scipy
>     NumPy version 1.9.0.dev-056ab73
>     NumPy is installed in
> /usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy
>     SciPy version 0.14.0
>     SciPy is installed in
> /usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy
>     Python version 2.7.5 (default, Jun 18 2013, 21:21:44) [GCC 4.2.1
> Compatible Apple LLVM 4.2 (clang-425.0.28)]
>     nose version 1.3.0
>     E...............EEEEEE............EEEEEEEEEE
>     ======================================================================
>     ERROR: Failure: ImportError
> (dlopen(/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so,
> 2): Symbol not found: _PyModule_Create2
>       Referenced from:
> /usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so
>       Expected in: flat namespace
>      in
> /usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so)
>     ----------------------------------------------------------------------
>     Traceback (most recent call last):
>       File
> "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/loader.py",
> line 413, in loadTestsFromName
>         addr.filename, addr.module)
>       File
> "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/importer.py",
> line 47, in importFromPath
>         return self.importFromDir(dir_path, fqname)
>       File
> "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/importer.py",
> line 94, in importFromDir
>         mod = load_module(part_fqname, fh, filename, desc)
>       File
> "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/__init__.py",
> line 27, in <module>
>         from . import vq, hierarchy
>       File
> "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/hierarchy.py",
> line 175, in <module>
>         from . import _hierarchy_wrap
>     ImportError:
> dlopen(/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so,
> 2): Symbol not found: _PyModule_Create2
>       Referenced from:
> /usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so
>       Expected in: flat namespace
>      in
> /usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so
>
> ...
> <42 more errors>
>

That is strange homebrew is one of tests in the grid and the installation
path looks strange.

Can you try downloading the wheel from the url and installing from the
local file?

Cheers,

Matthew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20140614/a75df5ba/attachment.html>