[Numpy-discussion] Unable to building numpy with openblas using bento or distutils
Skipper Seabold
jsseabold at gmail.com
Sat Mar 23 15:19:43 EDT 2013
On Sat, Mar 23, 2013 at 2:32 PM, Hanno Klemm <klemm at phys.ethz.ch> wrote:
> Skipper,
> this looks like a problem that I had in the bad old days with ATLAS, as
> well. Try compiling openblas with the -fPIC flag that used to help.
>
>
Thanks for having a look. I checked after seeing that odd bento failure
(see here [1]), and it looks to me like OpenBlas uses the -fPIC flag in all
of the gcc and gfortran calls. Possible related? [2]
Skipper
[1] https://github.com/cournape/Bento/issues/116
[2] https://github.com/cournape/Bento/issues/128
> Best of luck,
> Hanno
>
> hanno.klemm at me.com
>
> Sent from my mobile device, please excuse my brevity.
>
> On 23.03.2013, at 19:19, Skipper Seabold <jsseabold at gmail.com> wrote:
>
> Some help on this would be greatly appreciated. It's been recommended to
> use OpenBlas over ATLAS, so I've been trying to build numpy with openblas
> and have run into a few problems.
>
> 1) Build fails using bento master and waf 1.7.9, see below.
> 2) Distutils doesn't seem to be able to find lapack as part of atlas. I
> tried to skip a site.cfg and define environmental variables. No idea what I
> missed.
>
> I followed instructions found scattered over the internet and only
> understand vaguely the issues. Maybe someone can help. I'd be happy to
> update the wiki with any answers.
>
> To truly support OpenBlas, is it maybe necessary to make some additions to
> numpy/distutils/system_info.py?
>
> Thanks for having a look,
>
> Skipper
>
> Install OpenBlas
> -----------------------------
> git clone git://github.com/xianyi/OpenBLAS
> cd OpenBlas
>
> Edit c_check to look for libpthreads in the right place (Kubuntu 12.10)
>
> |4 $ git diff c_check
> ```
> diff --git a/c_check b/c_check
> index 4d82237..de0fd33 100644
> --- a/c_check
> +++ b/c_check
> @@ -241,7 +241,7 @@ print CONFFILE "#define FUNDERSCORE\t$need_fu\n" if
> $need_fu
>
> if ($os eq "LINUX") {
>
> - @pthread = split(/\s+/, `nm /lib/libpthread.so* | grep
> _pthread_create`);
> + @pthread = split(/\s+/, `nm /lib/x86_64-linux-gnu/libpthread.so* |
> grep _pthread_create`);
>
> if ($pthread[2] ne "") {
> print CONFFILE "#define PTHREAD_CREATE_FUNC $pthread[2]\n";
> ```
>
> make fc=gfortran
> make PREFIX=~/.local install
>
> Everything looks ok, so far.
>
> Install NumPy
> ---------------------------
> Using numpy master
>
> I tried to use bento master and waf 1.7.9, following instructions from
> David's blog
>
> bentomaker configure --prefix=/home/skipper/.local
> --with-blas-lapack-libdir=/home/skipper/.local/lib
> --blas-lapack-type=openblas ..
> bentomaker build -j4
>
> ```
> <snip>
> [101/104] cshlib: build/numpy/core/src/umath/umath_tests.c.11.o ->
> build/numpy/core/umath_tests.so
>
> /usr/bin/ld: numpy/core/lib/libnpymath.a(halffloat.c.12.o): relocation
> R_X86_64_PC32 against symbol `npy_halfbits_to_floatbits' can not be used
> when making a shared object; recompile with -fPIC
> /usr/bin/ld: final link failed: Bad value
> collect2: error: ld returned 1 exit status
> /usr/bin/ld: numpy/core/lib/libnpymath.a(halffloat.c.12.o): relocation
> R_X86_64_PC32 against symbol `npy_halfbits_to_floatbits' can not be used
> when making a shared object; recompile with -fPIC
> /usr/bin/ld: final link failed: Bad value
> collect2: error: ld returned 1 exit status
> ```
>
> No idea, so, let's try distutils
>
> export LAPACK=~/.local/lib/libopenblas.a
> export BLAS=~/.local/lib/libopenblas.a
> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/.local/lib/
> echo $LD_LIBRARY_PATH
> ```
> :/usr/local/lib64/R/bin:/home/skipper/.local/lib/
> ```
>
> This step seems to be necessary?
>
> python setup.py config
> ```
> Running from numpy source directory.
> non-existing path in 'numpy/distutils': 'site.cfg'
> F2PY Version 2
> numpy/core/setup_common.py:88: MismatchCAPIWarning: API mismatch detected,
> the C API version numbers have to be updated. Current C api version is 8,
> with checksum f4362353e2d72f889fda0128aa015037, but recorded checksum for C
> API version 8 in codegen_dir/cversions.txt is
> 17321775fc884de0b1eda478cd61c74b. If functions were added in the C API, you
> have to update C_API_VERSION in numpy/core/setup_common.py.
> MismatchCAPIWarning)
> blas_opt_info:
> blas_mkl_info:
> libraries mkl,vml,guide not found in ['/usr/local/lib64',
> '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
> NOT AVAILABLE
>
> atlas_blas_threads_info:
> Setting PTATLAS=ATLAS
> libraries ptf77blas,ptcblas,atlas not found in ['/usr/local/lib64',
> '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
> NOT AVAILABLE
>
> atlas_blas_info:
> libraries f77blas,cblas,atlas not found in ['/usr/local/lib64',
> '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
> NOT AVAILABLE
>
> /home/skipper/src/numpy-skipper/numpy/distutils/system_info.py:1501:
> UserWarning:
> Atlas (http://math-atlas.sourceforge.net/) libraries not found.
> Directories to search for the libraries can be specified in the
> numpy/distutils/site.cfg file (section [atlas]) or by setting
> the ATLAS environment variable.
> warnings.warn(AtlasNotFoundError.__doc__)
> blas_info:
> Replacing _lib_names[0]=='blas' with 'openblas'
> Replacing _lib_names[0]=='openblas' with 'openblas'
> FOUND:
> libraries = ['openblas']
> library_dirs = ['/home/skipper/.local/lib']
> language = f77
>
> FOUND:
> libraries = ['openblas']
> library_dirs = ['/home/skipper/.local/lib']
> define_macros = [('NO_ATLAS_INFO', 1)]
> language = f77
>
> non-existing path in 'numpy/lib': 'benchmarks'
> lapack_opt_info:
> lapack_mkl_info:
> mkl_info:
> libraries mkl,vml,guide not found in ['/usr/local/lib64',
> '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
> NOT AVAILABLE
>
> NOT AVAILABLE
>
> atlas_threads_info:
> Setting PTATLAS=ATLAS
> libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib64
> libraries lapack_atlas not found in /usr/local/lib64
> libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
> libraries lapack_atlas not found in /usr/local/lib
> libraries ptf77blas,ptcblas,atlas not found in /usr/lib64
> libraries lapack_atlas not found in /usr/lib64
> libraries ptf77blas,ptcblas,atlas not found in /usr/lib
> libraries lapack_atlas not found in /usr/lib
> libraries ptf77blas,ptcblas,atlas not found in /usr/lib/x86_64-linux-gnu
> libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
> numpy.distutils.system_info.atlas_threads_info
> NOT AVAILABLE
>
> atlas_info:
> libraries f77blas,cblas,atlas not found in /usr/local/lib64
> libraries lapack_atlas not found in /usr/local/lib64
> libraries f77blas,cblas,atlas not found in /usr/local/lib
> libraries lapack_atlas not found in /usr/local/lib
> libraries f77blas,cblas,atlas not found in /usr/lib64
> libraries lapack_atlas not found in /usr/lib64
> libraries f77blas,cblas,atlas not found in /usr/lib
> libraries lapack_atlas not found in /usr/lib
> libraries f77blas,cblas,atlas not found in /usr/lib/x86_64-linux-gnu
> libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
> numpy.distutils.system_info.atlas_info
> NOT AVAILABLE
>
> /home/skipper/src/numpy-skipper/numpy/distutils/system_info.py:1415:
> UserWarning:
> Atlas (http://math-atlas.sourceforge.net/) libraries not found.
> Directories to search for the libraries can be specified in the
> numpy/distutils/site.cfg file (section [atlas]) or by setting
> the ATLAS environment variable.
> warnings.warn(AtlasNotFoundError.__doc__)
> lapack_info:
> Replacing _lib_names[0]=='lapack' with 'openblas'
> Replacing _lib_names[0]=='openblas' with 'openblas'
> FOUND:
> libraries = ['openblas']
> library_dirs = ['/home/skipper/.local/lib']
> language = f77
>
> FOUND:
> libraries = ['openblas', 'openblas']
> library_dirs = ['/home/skipper/.local/lib']
> define_macros = [('NO_ATLAS_INFO', 1)]
> language = f77
>
> running config
> ```
>
> python setup.py build &> build.log
>
> Build log is here. Obviously it didn't go well, but I don't see anything
> to indicate problems. Sometimes I am able to get _dotblas.so built, though
> I don't know what causes it. This time I wasn't.
>
> https://gist.github.com/jseabold/7054ba9d85eae09eb402#file-numpy_build-log
>
> sudo python setup.py install &> install.log
>
>
> https://gist.github.com/jseabold/a0f5638b65d44aeff598#file-numpy_install-log
>
> >>> import numpy as np
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "/usr/local/lib/python2.7/dist-packages/numpy/__init__.py", line
> 138, in <module>
> import add_newdocs
> File "/usr/local/lib/python2.7/dist-packages/numpy/add_newdocs.py", line
> 13, in <module>
> from numpy.lib import add_newdoc
> File "/usr/local/lib/python2.7/dist-packages/numpy/lib/__init__.py",
> line 15, in <module>
> from polynomial import *
> File "/usr/local/lib/python2.7/dist-packages/numpy/lib/polynomial.py",
> line 19, in <module>
> from numpy.linalg import eigvals, lstsq, inv
> File "/usr/local/lib/python2.7/dist-packages/numpy/linalg/__init__.py",
> line 50, in <module>
> from linalg import *
> File "/usr/local/lib/python2.7/dist-packages/numpy/linalg/linalg.py",
> line 25, in <module>
> from numpy.linalg import lapack_lite
> ImportError: libopenblas.so.0: cannot open shared object file: No such
> file or directory
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130323/aa4a86a2/attachment.html>
More information about the NumPy-Discussion
mailing list