[Numpy-discussion] Switch to using ATLAS for OSX binary wheels

[Ralf Gommers]

On Fri, Jun 13, 2014 at 2:07 PM, Matthew Brett <matthew.brett at gmail.com>
wrote:

> Hi,
>
> Summary : I'm planning to upload OSX wheels for numpy and scipy using
> the ATLAS blas / lapack library instead of the default OSX Accelerate
> framework.
>
> We've run into some trouble with a segfault in recent OSX Accelerate:
>
> https://github.com/numpy/numpy/issues/4007
>
> and Accelerate also doesn't play well with multiprocessing.
>
> https://github.com/numpy/numpy/issues/4776
>
> Because there's nothing I love more than half-day compilation runs on
> my laptop, I've built ATLAS binaries with gcc 4.8, and linked numpy
> and scipy to them to make OSX wheels.  These pass all tests in i386
> and x86_64 mode, including numpy, scipy, matplotlib, pandas:
>
> https://travis-ci.org/matthew-brett/scipy-stack-osx-testing/builds/27442987
>
> The build process needs some automating, but it's recorded here:
>
> https://github.com/matthew-brett/numpy-atlas-binaries
>
> It's possible to get travis-ci to build these guys from a bare machine
> and then upload them somewhere, but I haven't tried to do that.
>
> Meanwhile Sturla kindly worked up a patch to numpy to work round the
> Accelerate segfault [1].  I haven't tested that, but given I'd already
> built the wheels, I prefer the ATLAS builds because they work with
> multiprocessing.
>
> I propose uploading these wheels as the default for numpy and scipy.
> Does anyone have any objection or comments before I go ahead and do
> that?
>

>From your README and wscript I don't see what numpy version you're using to
compile scipy against. I got the impression that you used 1.8.1, but it
should be numpy 1.5.1 for the 2.7 build, and 1.7.1 for 3.x.

I've tried the scipy 0.14.0 python2.7 wheel, but I get import errors (see
below). Your wheels should work with all common Python installs (mine is
homebrew) right?

Ralf


    $ python2.7 -c "import scipy; scipy.test()"
    Running unit tests for scipy
    NumPy version 1.9.0.dev-056ab73
    NumPy is installed in
/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy
    SciPy version 0.14.0
    SciPy is installed in
/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy
    Python version 2.7.5 (default, Jun 18 2013, 21:21:44) [GCC 4.2.1
Compatible Apple LLVM 4.2 (clang-425.0.28)]
    nose version 1.3.0
    E...............EEEEEE............EEEEEEEEEE
    ======================================================================
    ERROR: Failure: ImportError
(dlopen(/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so,
2): Symbol not found: _PyModule_Create2
      Referenced from:
/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so
      Expected in: flat namespace
     in
/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/loader.py",
line 413, in loadTestsFromName
        addr.filename, addr.module)
      File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/importer.py",
line 47, in importFromPath
        return self.importFromDir(dir_path, fqname)
      File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/importer.py",
line 94, in importFromDir
        mod = load_module(part_fqname, fh, filename, desc)
      File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/__init__.py",
line 27, in <module>
        from . import vq, hierarchy
      File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/hierarchy.py",
line 175, in <module>
        from . import _hierarchy_wrap
    ImportError:
dlopen(/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so,
2): Symbol not found: _PyModule_Create2
      Referenced from:
/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so
      Expected in: flat namespace
     in
/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so

...
<42 more errors>
...


> Cheers,
>
> Matthew
>
> [1] http://mail.scipy.org/pipermail/numpy-discussion/2014-June/070333.html
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20140613/8d10c274/attachment.html>