On Sat, Mar 23, 2013 at 2:32 PM, Hanno Klemm <span dir="ltr"><<a href="mailto:klemm@phys.ethz.ch" target="_blank">klemm@phys.ethz.ch</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="auto"><div>Skipper,</div><div>this looks like a problem that I had in the bad old days with ATLAS, as well. Try compiling openblas with the -fPIC flag that used to help. </div><div><br></div></div></blockquote>
<div>
<br></div><div>Thanks for having a look. I checked after seeing that odd bento failure (see here [1]), and it looks to me like OpenBlas uses the -fPIC flag in all of the gcc and gfortran calls. Possible related? [2]</div>
<div><br></div><div>Skipper</div><div><br></div><div>[1] <a href="https://github.com/cournape/Bento/issues/116">https://github.com/cournape/Bento/issues/116</a></div><div>[2] <a href="https://github.com/cournape/Bento/issues/128">https://github.com/cournape/Bento/issues/128</a></div>
<div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto"><div></div><div>Best of luck,</div><div>Hanno<br><br><a href="mailto:hanno.klemm@me.com" target="_blank">hanno.klemm@me.com</a><div>
<br></div><div>Sent from my mobile device, please excuse my brevity. </div></div><div><div class="h5"><div><br>On 23.03.2013, at 19:19, Skipper Seabold <<a href="mailto:jsseabold@gmail.com" target="_blank">jsseabold@gmail.com</a>> wrote:<br>
<br></div><blockquote type="cite"><div>Some help on this would be greatly appreciated. It's been recommended to use OpenBlas over ATLAS, so I've been trying to build numpy with openblas and have run into a few problems.<div>
<br></div><div>
<div>1) Build fails using bento master and waf 1.7.9, see below.</div><div>2) Distutils doesn't seem to be able to find lapack as part of atlas. I tried to skip a site.cfg and define environmental variables. No idea what I missed.</div>
<div><br></div><div>I followed instructions found scattered over the internet and only understand vaguely the issues. Maybe someone can help. I'd be happy to update the wiki with any answers. </div>
<div><br></div><div>To truly support OpenBlas, is it maybe necessary to make some additions to numpy/distutils/system_info.py?</div></div><div><br></div><div>Thanks for having a look,</div><div><br></div><div>Skipper</div>
<div><br></div><div>Install OpenBlas</div><div>-----------------------------</div><div><div>
git clone git://<a href="http://github.com/xianyi/OpenBLAS" target="_blank">github.com/xianyi/OpenBLAS</a></div><div>cd OpenBlas</div><div><br></div><div>Edit c_check to look for libpthreads in the right place (Kubuntu 12.10)</div>
<div><br></div><div>|4 $ git diff c_check</div>
<div>```</div><div>diff --git a/c_check b/c_check</div><div>index 4d82237..de0fd33 100644</div><div>--- a/c_check</div><div>+++ b/c_check</div><div>@@ -241,7 +241,7 @@ print CONFFILE "#define FUNDERSCORE\t$need_fu\n" if $need_fu</div>
<div> </div><div> if ($os eq "LINUX") {</div><div> </div><div>- @pthread = split(/\s+/, `nm /lib/libpthread.so* | grep _pthread_create`);</div><div>+ @pthread = split(/\s+/, `nm /lib/x86_64-linux-gnu/libpthread.so* | grep _pthread_create`);</div>
<div> </div><div> if ($pthread[2] ne "") {</div><div> print CONFFILE "#define PTHREAD_CREATE_FUNC $pthread[2]\n";</div><div>```</div><div><br></div><div>make fc=gfortran</div><div>make PREFIX=~/.local install</div>
</div><div><br></div><div>Everything looks ok, so far. </div><div><br></div><div>Install NumPy</div><div>---------------------------</div><div>Using numpy master</div><div><br></div><div>I tried to use bento master and waf 1.7.9, following instructions from David's blog</div>
<div>
<br></div><div><div>bentomaker configure --prefix=/home/skipper/.local --with-blas-lapack-libdir=/home/skipper/.local/lib --blas-lapack-type=openblas ..</div><div>bentomaker build -j4</div><div><br></div><div>```</div><div>
<snip></div><div>[101/104] cshlib: build/numpy/core/src/umath/umath_tests.c.11.o -> build/numpy/core/umath_tests.so </div><div>/usr/bin/ld: numpy/core/lib/libnpymath.a(halffloat.c.12.o): relocation R_X86_64_PC32 against symbol `npy_halfbits_to_floatbits' can not be used when making a shared object; recompile with -fPIC</div>
<div>/usr/bin/ld: final link failed: Bad value</div><div>collect2: error: ld returned 1 exit status</div><div>/usr/bin/ld: numpy/core/lib/libnpymath.a(halffloat.c.12.o): relocation R_X86_64_PC32 against symbol `npy_halfbits_to_floatbits' can not be used when making a shared object; recompile with -fPIC</div>
<div>/usr/bin/ld: final link failed: Bad value</div><div>collect2: error: ld returned 1 exit status</div><div>```</div></div><div><br></div><div>No idea, so, let's try distutils</div><div><br></div><div><div>export LAPACK=~/.local/lib/libopenblas.a</div>
<div>export BLAS=~/.local/lib/libopenblas.a</div></div><div><div>export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/.local/lib/</div></div><div>echo $LD_LIBRARY_PATH</div><div>```</div><div>:/usr/local/lib64/R/bin:/home/skipper/.local/lib/</div>
<div>```</div><div><br></div><div>This step seems to be necessary?</div><div><br></div><div><div>python setup.py config</div><div>```</div>
<div><div>Running from numpy source directory.</div><div>non-existing path in 'numpy/distutils': 'site.cfg'</div><div>F2PY Version 2</div><div>numpy/core/setup_common.py:88: MismatchCAPIWarning: API mismatch detected, the C API version numbers have to be updated. Current C api version is 8, with checksum f4362353e2d72f889fda0128aa015037, but recorded checksum for C API version 8 in codegen_dir/cversions.txt is 17321775fc884de0b1eda478cd61c74b. If functions were added in the C API, you have to update C_API_VERSION in numpy/core/setup_common.py.</div>
<div> MismatchCAPIWarning)</div><div>blas_opt_info:</div><div>blas_mkl_info:</div><div> libraries mkl,vml,guide not found in ['/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']</div>
<div> NOT AVAILABLE</div><div><br></div><div>atlas_blas_threads_info:</div><div>Setting PTATLAS=ATLAS</div><div> libraries ptf77blas,ptcblas,atlas not found in ['/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']</div>
<div> NOT AVAILABLE</div><div><br></div><div>atlas_blas_info:</div><div> libraries f77blas,cblas,atlas not found in ['/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']</div>
<div> NOT AVAILABLE</div><div><br></div><div>/home/skipper/src/numpy-skipper/numpy/distutils/system_info.py:1501: UserWarning: </div><div> Atlas (<a href="http://math-atlas.sourceforge.net/" target="_blank">http://math-atlas.sourceforge.net/</a>) libraries not found.</div>
<div> Directories to search for the libraries can be specified in the</div><div> numpy/distutils/site.cfg file (section [atlas]) or by setting</div><div> the ATLAS environment variable.</div><div> warnings.warn(AtlasNotFoundError.__doc__)</div>
<div>blas_info:</div><div>Replacing _lib_names[0]=='blas' with 'openblas'</div><div>Replacing _lib_names[0]=='openblas' with 'openblas'</div><div> FOUND:</div><div> libraries = ['openblas']</div>
<div> library_dirs = ['/home/skipper/.local/lib']</div><div> language = f77</div><div><br></div><div> FOUND:</div><div> libraries = ['openblas']</div><div> library_dirs = ['/home/skipper/.local/lib']</div>
<div> define_macros = [('NO_ATLAS_INFO', 1)]</div><div> language = f77</div><div><br></div><div>non-existing path in 'numpy/lib': 'benchmarks'</div><div>lapack_opt_info:</div><div>lapack_mkl_info:</div>
<div>mkl_info:</div><div> libraries mkl,vml,guide not found in ['/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']</div><div> NOT AVAILABLE</div>
<div><br></div><div> NOT AVAILABLE</div><div><br></div><div>atlas_threads_info:</div><div>Setting PTATLAS=ATLAS</div><div> libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib64</div><div> libraries lapack_atlas not found in /usr/local/lib64</div>
<div> libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib</div><div> libraries lapack_atlas not found in /usr/local/lib</div><div> libraries ptf77blas,ptcblas,atlas not found in /usr/lib64</div><div> libraries lapack_atlas not found in /usr/lib64</div>
<div> libraries ptf77blas,ptcblas,atlas not found in /usr/lib</div><div> libraries lapack_atlas not found in /usr/lib</div><div> libraries ptf77blas,ptcblas,atlas not found in /usr/lib/x86_64-linux-gnu</div><div> libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu</div>
<div>numpy.distutils.system_info.atlas_threads_info</div><div> NOT AVAILABLE</div><div><br></div><div>atlas_info:</div><div> libraries f77blas,cblas,atlas not found in /usr/local/lib64</div><div> libraries lapack_atlas not found in /usr/local/lib64</div>
<div> libraries f77blas,cblas,atlas not found in /usr/local/lib</div><div> libraries lapack_atlas not found in /usr/local/lib</div><div> libraries f77blas,cblas,atlas not found in /usr/lib64</div><div> libraries lapack_atlas not found in /usr/lib64</div>
<div> libraries f77blas,cblas,atlas not found in /usr/lib</div><div> libraries lapack_atlas not found in /usr/lib</div><div> libraries f77blas,cblas,atlas not found in /usr/lib/x86_64-linux-gnu</div><div> libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu</div>
<div>numpy.distutils.system_info.atlas_info</div><div> NOT AVAILABLE</div><div><br></div><div>/home/skipper/src/numpy-skipper/numpy/distutils/system_info.py:1415: UserWarning: </div><div> Atlas (<a href="http://math-atlas.sourceforge.net/" target="_blank">http://math-atlas.sourceforge.net/</a>) libraries not found.</div>
<div> Directories to search for the libraries can be specified in the</div><div> numpy/distutils/site.cfg file (section [atlas]) or by setting</div><div> the ATLAS environment variable.</div><div> warnings.warn(AtlasNotFoundError.__doc__)</div>
<div>lapack_info:</div><div>Replacing _lib_names[0]=='lapack' with 'openblas'</div><div>Replacing _lib_names[0]=='openblas' with 'openblas'</div><div> FOUND:</div><div> libraries = ['openblas']</div>
<div> library_dirs = ['/home/skipper/.local/lib']</div><div> language = f77</div><div><br></div><div> FOUND:</div><div> libraries = ['openblas', 'openblas']</div><div> library_dirs = ['/home/skipper/.local/lib']</div>
<div> define_macros = [('NO_ATLAS_INFO', 1)]</div><div> language = f77</div><div><br></div><div>running config</div></div><div>```</div></div><div><br></div><div><div>python setup.py build &> build.log</div>
</div><div><br></div><div>Build log is here. Obviously it didn't go well, but I don't see anything to indicate problems. Sometimes I am able to get _dotblas.so built, though I don't know what causes it. This time I wasn't.</div>
<div><br></div><div><a href="https://gist.github.com/jseabold/7054ba9d85eae09eb402#file-numpy_build-log" target="_blank">https://gist.github.com/jseabold/7054ba9d85eae09eb402#file-numpy_build-log</a></div>
<div><br></div><div>sudo python setup.py install &> install.log</div><div><br></div><div><a href="https://gist.github.com/jseabold/a0f5638b65d44aeff598#file-numpy_install-log" target="_blank">https://gist.github.com/jseabold/a0f5638b65d44aeff598#file-numpy_install-log</a></div>
<div><br></div><div><div>>>> import numpy as np</div><div>Traceback (most recent call last):</div><div> File "<stdin>", line 1, in <module></div><div> File "/usr/local/lib/python2.7/dist-packages/numpy/__init__.py", line 138, in <module></div>
<div> import add_newdocs</div><div> File "/usr/local/lib/python2.7/dist-packages/numpy/add_newdocs.py", line 13, in <module></div><div> from numpy.lib import add_newdoc</div><div> File "/usr/local/lib/python2.7/dist-packages/numpy/lib/__init__.py", line 15, in <module></div>
<div> from polynomial import *</div><div> File "/usr/local/lib/python2.7/dist-packages/numpy/lib/polynomial.py", line 19, in <module></div><div> from numpy.linalg import eigvals, lstsq, inv</div><div>
File "/usr/local/lib/python2.7/dist-packages/numpy/linalg/__init__.py", line 50, in <module></div><div> from linalg import *</div><div> File "/usr/local/lib/python2.7/dist-packages/numpy/linalg/linalg.py", line 25, in <module></div>
<div> from numpy.linalg import lapack_lite</div><div>ImportError: libopenblas.so.0: cannot open shared object file: No such file or directory</div></div>
</div></blockquote></div></div><blockquote type="cite"><div><span>_______________________________________________</span><br><span>NumPy-Discussion mailing list</span><br><span><a href="mailto:NumPy-Discussion@scipy.org" target="_blank">NumPy-Discussion@scipy.org</a></span><br>
<span><a href="http://mail.scipy.org/mailman/listinfo/numpy-discussion" target="_blank">http://mail.scipy.org/mailman/listinfo/numpy-discussion</a></span><br></div></blockquote></div><br>_______________________________________________<br>
NumPy-Discussion mailing list<br>
<a href="mailto:NumPy-Discussion@scipy.org">NumPy-Discussion@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/numpy-discussion" target="_blank">http://mail.scipy.org/mailman/listinfo/numpy-discussion</a><br>
<br></blockquote></div><br>