[Numpy-discussion] Switch to using ATLAS for OSX binary wheels

[Ralf Gommers]

On Sat, Jun 14, 2014 at 11:56 AM, Matthew Brett <matthew.brett at gmail.com>
wrote:

>
>
> On Friday, June 13, 2014, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>>
>>
>>
>> On Fri, Jun 13, 2014 at 2:07 PM, Matthew Brett <matthew.brett at gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Summary : I'm planning to upload OSX wheels for numpy and scipy using
>>> the ATLAS blas / lapack library instead of the default OSX Accelerate
>>> framework.
>>>
>>> We've run into some trouble with a segfault in recent OSX Accelerate:
>>>
>>> https://github.com/numpy/numpy/issues/4007
>>>
>>> and Accelerate also doesn't play well with multiprocessing.
>>>
>>> https://github.com/numpy/numpy/issues/4776
>>>
>>> Because there's nothing I love more than half-day compilation runs on
>>> my laptop, I've built ATLAS binaries with gcc 4.8, and linked numpy
>>> and scipy to them to make OSX wheels.  These pass all tests in i386
>>> and x86_64 mode, including numpy, scipy, matplotlib, pandas:
>>>
>>>
>>> https://travis-ci.org/matthew-brett/scipy-stack-osx-testing/builds/27442987
>>>
>>> The build process needs some automating, but it's recorded here:
>>>
>>> https://github.com/matthew-brett/numpy-atlas-binaries
>>>
>>> It's possible to get travis-ci to build these guys from a bare machine
>>> and then upload them somewhere, but I haven't tried to do that.
>>>
>>> Meanwhile Sturla kindly worked up a patch to numpy to work round the
>>> Accelerate segfault [1].  I haven't tested that, but given I'd already
>>> built the wheels, I prefer the ATLAS builds because they work with
>>> multiprocessing.
>>>
>>> I propose uploading these wheels as the default for numpy and scipy.
>>> Does anyone have any objection or comments before I go ahead and do
>>> that?
>>>
>>
>> From your README and wscript I don't see what numpy version you're using
>> to compile scipy against. I got the impression that you used 1.8.1, but it
>> should be numpy 1.5.1 for the 2.7 build, and 1.7.1 for 3.x.
>>
>> I've tried the scipy 0.14.0 python2.7 wheel, but I get import errors (see
>> below). Your wheels should work with all common Python installs (mine is
>> homebrew) right?
>>
>> Ralf
>>
>>
>>     $ python2.7 -c "import scipy; scipy.test()"
>>     Running unit tests for scipy
>>     NumPy version 1.9.0.dev-056ab73
>>     NumPy is installed in
>> /usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy
>>     SciPy version 0.14.0
>>     SciPy is installed in
>> /usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy
>>     Python version 2.7.5 (default, Jun 18 2013, 21:21:44) [GCC 4.2.1
>> Compatible Apple LLVM 4.2 (clang-425.0.28)]
>>     nose version 1.3.0
>>     E...............EEEEEE............EEEEEEEEEE
>>     ======================================================================
>>     ERROR: Failure: ImportError
>> (dlopen(/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so,
>> 2): Symbol not found: _PyModule_Create2
>>       Referenced from:
>> /usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so
>>       Expected in: flat namespace
>>      in
>> /usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so)
>>     ----------------------------------------------------------------------
>>     Traceback (most recent call last):
>>       File
>> "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/loader.py",
>> line 413, in loadTestsFromName
>>         addr.filename, addr.module)
>>       File
>> "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/importer.py",
>> line 47, in importFromPath
>>         return self.importFromDir(dir_path, fqname)
>>       File
>> "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/importer.py",
>> line 94, in importFromDir
>>         mod = load_module(part_fqname, fh, filename, desc)
>>       File
>> "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/__init__.py",
>> line 27, in <module>
>>         from . import vq, hierarchy
>>       File
>> "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/hierarchy.py",
>> line 175, in <module>
>>         from . import _hierarchy_wrap
>>     ImportError:
>> dlopen(/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so,
>> 2): Symbol not found: _PyModule_Create2
>>       Referenced from:
>> /usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so
>>       Expected in: flat namespace
>>      in
>> /usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so
>>
>> ...
>> <42 more errors>
>>
>
> That is strange homebrew is one of tests in the grid and the installation
> path looks strange.
>
> Can you try downloading the wheel from the url and installing from the
> local file?
>

That's what I did (easier than remembering the magic pip incantation). The
install path looks fine to me, maybe homebrew changed it recently? I can
try to update my install, will take a few days though.

Ralf


>
> Cheers,
>
> Matthew
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20140615/2fbe1d53/attachment.html>