On Sat, Mar 23, 2013 at 2:32 PM, Hanno Klemm wrote:

Skipper, this looks like a problem that I had in the bad old days with ATLAS, as well. Try compiling openblas with the -fPIC flag that used to help.

Thanks for having a look. I checked after seeing that odd bento failure (see here [1]), and it looks to me like OpenBlas uses the -fPIC flag in all of the gcc and gfortran calls. Possible related? [2] Skipper [1] https://github.com/cournape/Bento/issues/116 [2] https://github.com/cournape/Bento/issues/128

Best of luck, Hanno

hanno.klemm@me.com

Sent from my mobile device, please excuse my brevity.

On 23.03.2013, at 19:19, Skipper Seabold wrote:

Some help on this would be greatly appreciated. It's been recommended to use OpenBlas over ATLAS, so I've been trying to build numpy with openblas and have run into a few problems.

1) Build fails using bento master and waf 1.7.9, see below. 2) Distutils doesn't seem to be able to find lapack as part of atlas. I tried to skip a site.cfg and define environmental variables. No idea what I missed.

I followed instructions found scattered over the internet and only understand vaguely the issues. Maybe someone can help. I'd be happy to update the wiki with any answers.

To truly support OpenBlas, is it maybe necessary to make some additions to numpy/distutils/system_info.py?

Thanks for having a look,

Skipper

Install OpenBlas ----------------------------- git clone git://github.com/xianyi/OpenBLAS cd OpenBlas

Edit c_check to look for libpthreads in the right place (Kubuntu 12.10)

|4 $ git diff c_check ``` diff --git a/c_check b/c_check index 4d82237..de0fd33 100644 --- a/c_check +++ b/c_check @@ -241,7 +241,7 @@ print CONFFILE "#define FUNDERSCORE\t$need_fu\n" if $need_fu

if ($os eq "LINUX") {

- @pthread = split(/\s+/, `nm /lib/libpthread.so* | grep _pthread_create`); + @pthread = split(/\s+/, `nm /lib/x86_64-linux-gnu/libpthread.so* | grep _pthread_create`);

if ($pthread[2] ne "") { print CONFFILE "#define PTHREAD_CREATE_FUNC $pthread[2]\n"; ```

make fc=gfortran make PREFIX=~/.local install

Everything looks ok, so far.

Install NumPy --------------------------- Using numpy master

I tried to use bento master and waf 1.7.9, following instructions from David's blog

bentomaker configure --prefix=/home/skipper/.local --with-blas-lapack-libdir=/home/skipper/.local/lib --blas-lapack-type=openblas .. bentomaker build -j4

``` <snip> [101/104] cshlib: build/numpy/core/src/umath/umath_tests.c.11.o -> build/numpy/core/umath_tests.so

/usr/bin/ld: numpy/core/lib/libnpymath.a(halffloat.c.12.o): relocation R_X86_64_PC32 against symbol `npy_halfbits_to_floatbits' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: final link failed: Bad value collect2: error: ld returned 1 exit status /usr/bin/ld: numpy/core/lib/libnpymath.a(halffloat.c.12.o): relocation R_X86_64_PC32 against symbol `npy_halfbits_to_floatbits' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: final link failed: Bad value collect2: error: ld returned 1 exit status ```

No idea, so, let's try distutils

export LAPACK=~/.local/lib/libopenblas.a export BLAS=~/.local/lib/libopenblas.a export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/.local/lib/ echo $LD_LIBRARY_PATH ``` :/usr/local/lib64/R/bin:/home/skipper/.local/lib/ ```

This step seems to be necessary?

python setup.py config ``` Running from numpy source directory. non-existing path in 'numpy/distutils': 'site.cfg' F2PY Version 2 numpy/core/setup_common.py:88: MismatchCAPIWarning: API mismatch detected, the C API version numbers have to be updated. Current C api version is 8, with checksum f4362353e2d72f889fda0128aa015037, but recorded checksum for C API version 8 in codegen_dir/cversions.txt is 17321775fc884de0b1eda478cd61c74b. If functions were added in the C API, you have to update C_API_VERSION in numpy/core/setup_common.py. MismatchCAPIWarning) blas_opt_info: blas_mkl_info: libraries mkl,vml,guide not found in ['/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu'] NOT AVAILABLE

atlas_blas_threads_info: Setting PTATLAS=ATLAS libraries ptf77blas,ptcblas,atlas not found in ['/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu'] NOT AVAILABLE

atlas_blas_info: libraries f77blas,cblas,atlas not found in ['/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu'] NOT AVAILABLE

/home/skipper/src/numpy-skipper/numpy/distutils/system_info.py:1501: UserWarning: Atlas (http://math-atlas.sourceforge.net/) libraries not found. Directories to search for the libraries can be specified in the numpy/distutils/site.cfg file (section [atlas]) or by setting the ATLAS environment variable. warnings.warn(AtlasNotFoundError.__doc__) blas_info: Replacing _lib_names[0]=='blas' with 'openblas' Replacing _lib_names[0]=='openblas' with 'openblas' FOUND: libraries = ['openblas'] library_dirs = ['/home/skipper/.local/lib'] language = f77

FOUND: libraries = ['openblas'] library_dirs = ['/home/skipper/.local/lib'] define_macros = [('NO_ATLAS_INFO', 1)] language = f77

non-existing path in 'numpy/lib': 'benchmarks' lapack_opt_info: lapack_mkl_info: mkl_info: libraries mkl,vml,guide not found in ['/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu'] NOT AVAILABLE

NOT AVAILABLE

atlas_threads_info: Setting PTATLAS=ATLAS libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib64 libraries lapack_atlas not found in /usr/local/lib64 libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib libraries lapack_atlas not found in /usr/local/lib libraries ptf77blas,ptcblas,atlas not found in /usr/lib64 libraries lapack_atlas not found in /usr/lib64 libraries ptf77blas,ptcblas,atlas not found in /usr/lib libraries lapack_atlas not found in /usr/lib libraries ptf77blas,ptcblas,atlas not found in /usr/lib/x86_64-linux-gnu libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu numpy.distutils.system_info.atlas_threads_info NOT AVAILABLE

atlas_info: libraries f77blas,cblas,atlas not found in /usr/local/lib64 libraries lapack_atlas not found in /usr/local/lib64 libraries f77blas,cblas,atlas not found in /usr/local/lib libraries lapack_atlas not found in /usr/local/lib libraries f77blas,cblas,atlas not found in /usr/lib64 libraries lapack_atlas not found in /usr/lib64 libraries f77blas,cblas,atlas not found in /usr/lib libraries lapack_atlas not found in /usr/lib libraries f77blas,cblas,atlas not found in /usr/lib/x86_64-linux-gnu libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu numpy.distutils.system_info.atlas_info NOT AVAILABLE

/home/skipper/src/numpy-skipper/numpy/distutils/system_info.py:1415: UserWarning: Atlas (http://math-atlas.sourceforge.net/) libraries not found. Directories to search for the libraries can be specified in the numpy/distutils/site.cfg file (section [atlas]) or by setting the ATLAS environment variable. warnings.warn(AtlasNotFoundError.__doc__) lapack_info: Replacing _lib_names[0]=='lapack' with 'openblas' Replacing _lib_names[0]=='openblas' with 'openblas' FOUND: libraries = ['openblas'] library_dirs = ['/home/skipper/.local/lib'] language = f77

FOUND: libraries = ['openblas', 'openblas'] library_dirs = ['/home/skipper/.local/lib'] define_macros = [('NO_ATLAS_INFO', 1)] language = f77

running config ```

python setup.py build &> build.log

Build log is here. Obviously it didn't go well, but I don't see anything to indicate problems. Sometimes I am able to get _dotblas.so built, though I don't know what causes it. This time I wasn't.

https://gist.github.com/jseabold/7054ba9d85eae09eb402#file-numpy_build-log

sudo python setup.py install &> install.log

https://gist.github.com/jseabold/a0f5638b65d44aeff598#file-numpy_install-log

...

...

...

import numpy as np Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.7/dist-packages/numpy/__init__.py", line 138, in <module> import add_newdocs File "/usr/local/lib/python2.7/dist-packages/numpy/add_newdocs.py", line 13, in <module> from numpy.lib import add_newdoc File "/usr/local/lib/python2.7/dist-packages/numpy/lib/__init__.py", line 15, in <module> from polynomial import * File "/usr/local/lib/python2.7/dist-packages/numpy/lib/polynomial.py", line 19, in <module> from numpy.linalg import eigvals, lstsq, inv File "/usr/local/lib/python2.7/dist-packages/numpy/linalg/__init__.py", line 50, in <module> from linalg import * File "/usr/local/lib/python2.7/dist-packages/numpy/linalg/linalg.py", line 25, in <module> from numpy.linalg import lapack_lite ImportError: libopenblas.so.0: cannot open shared object file: No such file or directory

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion