Hi, On Mon, Jun 16, 2014 at 12:51 PM, <josef.pktd@gmail.com> wrote:
On Mon, Jun 16, 2014 at 7:10 AM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
On Sun, Jun 15, 2014 at 10:51 PM, Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Sat, Jun 14, 2014 at 11:56 AM, Matthew Brett <matthew.brett@gmail.com> wrote:
On Friday, June 13, 2014, Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Fri, Jun 13, 2014 at 2:07 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
Summary : I'm planning to upload OSX wheels for numpy and scipy using the ATLAS blas / lapack library instead of the default OSX Accelerate framework.
We've run into some trouble with a segfault in recent OSX Accelerate:
https://github.com/numpy/numpy/issues/4007
and Accelerate also doesn't play well with multiprocessing.
https://github.com/numpy/numpy/issues/4776
Because there's nothing I love more than half-day compilation runs on my laptop, I've built ATLAS binaries with gcc 4.8, and linked numpy and scipy to them to make OSX wheels. These pass all tests in i386 and x86_64 mode, including numpy, scipy, matplotlib, pandas:
https://travis-ci.org/matthew-brett/scipy-stack-osx-testing/builds/27442987
The build process needs some automating, but it's recorded here:
https://github.com/matthew-brett/numpy-atlas-binaries
It's possible to get travis-ci to build these guys from a bare machine and then upload them somewhere, but I haven't tried to do that.
Meanwhile Sturla kindly worked up a patch to numpy to work round the Accelerate segfault [1]. I haven't tested that, but given I'd already built the wheels, I prefer the ATLAS builds because they work with multiprocessing.
I propose uploading these wheels as the default for numpy and scipy. Does anyone have any objection or comments before I go ahead and do that?
From your README and wscript I don't see what numpy version you're using to compile scipy against. I got the impression that you used 1.8.1, but it should be numpy 1.5.1 for the 2.7 build, and 1.7.1 for 3.x.
I've tried the scipy 0.14.0 python2.7 wheel, but I get import errors (see below). Your wheels should work with all common Python installs (mine is homebrew) right?
Ralf
$ python2.7 -c "import scipy; scipy.test()" Running unit tests for scipy NumPy version 1.9.0.dev-056ab73 NumPy is installed in /usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy SciPy version 0.14.0 SciPy is installed in /usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy Python version 2.7.5 (default, Jun 18 2013, 21:21:44) [GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)] nose version 1.3.0 E...............EEEEEE............EEEEEEEEEE
====================================================================== ERROR: Failure: ImportError (dlopen(/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so, 2): Symbol not found: _PyModule_Create2 Referenced from: /usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so Expected in: flat namespace in /usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so)
---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/loader.py", line 413, in loadTestsFromName addr.filename, addr.module) File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/importer.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/importer.py", line 94, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/__init__.py", line 27, in <module> from . import vq, hierarchy File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/hierarchy.py", line 175, in <module> from . import _hierarchy_wrap ImportError: dlopen(/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so, 2): Symbol not found: _PyModule_Create2 Referenced from: /usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so Expected in: flat namespace in /usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/cluster/_hierarchy_wrap.so
... <42 more errors>
That is strange homebrew is one of tests in the grid and the installation path looks strange.
Can you try downloading the wheel from the url and installing from the local file?
That's what I did (easier than remembering the magic pip incantation). The install path looks fine to me, maybe homebrew changed it recently? I can try to update my install, will take a few days though.
Yes, sorry - that does look like the normal homebrew install path, I didn't realize it had the exotic framework parts to it.
I just ran these commands on my machine:
SPI=https://nipy.bic.berkeley.edu/scipy_installers BREW_BIN=/usr/local/Cellar/python/2.7.6/bin curl -O $SPI/numpy-1.8.1-cp27-none-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.whl curl -O $SPI/scipy-0.14.0-cp27-none-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.whl $BREW_BIN/pip install numpy-1.8.1-cp27-none-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.whl $BREW_BIN/pip install scipy-0.14.0-cp27-none-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.whl $BREW_BIN/python -c "import scipy; scipy.test()"
and the scipy tests passed.
I built the scipy wheel against numpy 1.8.1 - but - aren't the numpies binary compatible? What difference would I expect to see if I'd built scipy against numpy 1.5.1 or 1.7 ?
I summarized here what David explained a while ago about the difference between forward and backwards binary compatibility
http://stackoverflow.com/questions/17709641/valueerror-numpy-dtype-has-the-w...
I see - thanks for the summary. I will recompile. Cheers, Matthew