[Numpy-discussion] is my numpy installation using custom blas/lapack?

Sun Jun 7 00:11:10 EDT 2009

Hi,

On Jun 6, 2009 3:11pm, Chris Colbert <sccolbert at gmail.com> wrote:
> it definately found your threaded atlas libraries. How do you know

> it's numpy is using lapack_lite?

I don't, actually. But it is importing it. With python -v, this is the  
error I get if I don't set LD_LIBRARY_PATH to my scipy_build directory

import numpy.linalg.linalg # precompiled from  
/data10/users/rich/usr/galois/lib64/python/numpy/linalg/linalg.pyc
dlopen("/data10/users/rich/usr/galois/lib64/python/numpy/linalg/lapack_lite.so",  
2);
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/data10/users/rich/usr/galois//lib64/python/numpy/__init__.py", line  
130, in <module>
import add_newdocs
File "/data10/users/rich/usr/galois//lib64/python/numpy/add_newdocs.py",  
line 9, in <module>
from lib import add_newdoc
File "/data10/users/rich/usr/galois//lib64/python/numpy/lib/__init__.py",  
line 13, in <module>
from polynomial import *
File "/data10/users/rich/usr/galois//lib64/python/numpy/lib/polynomial.py",  
line 18, in <module>
from numpy.linalg import eigvals, lstsq
File "/data10/users/rich/usr/galois//lib64/python/numpy/linalg/__init__.py",  
line  
47, in <module>
from linalg import *
File "/data10/users/rich/usr/galois//lib64/python/numpy/linalg/linalg.py",  
line 22, in <module>
from numpy.linalg import lapack_lite
ImportError: liblapack.so: cannot open shared object file: No such file or  
directory

Here blas_opt_info seems to be missing ATLAS version.

>>> numpy.show_config()
atlas_threads_info:
libraries = ['lapack', 'lapack', 'f77blas', 'cblas', 'atlas']
library_dirs = ['/usr/local/rich/src/scipy_build/lib']
language = f77

blas_opt_info:
libraries = ['lapack', 'f77blas', 'cblas', 'atlas']
library_dirs = ['/usr/local/rich/src/scipy_build/lib']
define_macros = [('NO_ATLAS_INFO', 2)]
language = c

atlas_blas_threads_info:
libraries = ['lapack', 'f77blas', 'cblas', 'atlas']
library_dirs = ['/usr/local/rich/src/scipy_build/lib']
language = c

lapack_opt_info:
libraries = ['lapack', 'lapack', 'f77blas', 'cblas', 'atlas']
library_dirs = ['/usr/local/rich/src/scipy_build/lib']
define_macros = [('NO_ATLAS_INFO', 2)]
language = f77

lapack_mkl_info:
NOT AVAILABLE

blas_mkl_info:
NOT AVAILABLE

mkl_info:
NOT AVAILABLE

> when I do:

> python

> >>import numpy

> >>numpy.show_config()

> atlas_threads_info:

> libraries = ['lapack', 'ptf77blas', 'ptcblas', 'atlas']

> library_dirs = ['/usr/local/lib']

> language = f77

> blas_opt_info:

> libraries = ['ptf77blas', 'ptcblas', 'atlas']

> library_dirs = ['/usr/local/lib']

> define_macros = [('ATLAS_INFO', '"\\"3.8.3\\""')]

> language = c

> atlas_blas_threads_info:

> libraries = ['ptf77blas', 'ptcblas', 'atlas']

> library_dirs = ['/usr/local/lib']

> language = c

> lapack_opt_info:

> libraries = ['lapack', 'ptf77blas', 'ptcblas', 'atlas']

> library_dirs = ['/usr/local/lib']

> define_macros = [('NO_ATLAS_INFO', 2)]

> language = f77

> lapack_mkl_info:

> NOT AVAILABLE

> blas_mkl_info:

> NOT AVAILABLE

> mkl_info:

> NOT AVAILABLE

> also try:

> >>> a = numpy.random.randn(6000, 6000)

> >>> numpy.dot(a,a)

> and make sure all your cpu cores peg at 100%

Unfortunately only one cpu. What does that mean? Threaded libraries not  
used?

from top:

Cpu0 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu1 : 0.0%us, 0.2%sy, 0.0%ni, 99.4%id, 0.0%wa, 0.2%hi, 0.2%si, 0.0%st
Cpu2 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu3 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Thanks much for the help.

Rich

> On Sat, Jun 6, 2009 at 3:35 PM, llewelr at gmail.com> wrote:

> > Oops. Thanks, that makes more sense:

> >

> > http://pastebin.com/m7067709b

> >

> > On Jun 6, 2009 12:15pm, Chris Colbert sccolbert at gmail.com> wrote:

> >> i need the full link to pastebin.com in order to view your post.

> >>

> >>

> >>

> >> It will be something like http://pastebin.com/m6b09f05c

> >>

> >>

> >>

> >>

> >>

> >> chris

> >>

> >>

> >>

> >>

> >>

> >> On Sat, Jun 6, 2009 at 2:32 PM, Richard Llewellynllewelr at gmail.com>  
> wrote:

> >>

> >> > I posted the setup.py build output to pastebin.com, though missed the

> >>

> >> > uninteresting stderr (forgot tcsh command to redirect both).

> >>

> >> > Also, used setup.py build --fcompiler=gnu95.

> >>

> >> >

> >>

> >> >

> >>

> >> > To be clear, I am not certain that my ATLAS libraries are not found.  
> But

> >>

> >> > during the build starting at line 95 (pastebin.com) I see a  
> compilation

> >>

> >> > failure, and then NO_ATLAS_INFO, 2.

> >>

> >> >

> >>

> >> > I don't think I can use ldconfig without root, but have set

> >> > LD_LIBRARY_PATH

> >>

> >> > to point to the scipy_build/lib until I put them somewhere else.

> >>

> >> >

> >>

> >> > importing numpy works, though lapack_lite is also imported. I wonder  
> if

> >> > this

> >>

> >> > is normal even if my ATLAS was used.

> >>

> >> >

> >>

> >> > Thanks,

> >>

> >> > Rich

> >>

> >> >

> >>

> >> > On Sat, Jun 6, 2009 at 10:46 AM, Chris Colbert sccolbert at gmail.com>

> >> > wrote:

> >>

> >> >>

> >>

> >> >> and where exactly are you seeing atlas not found? during the build

> >>

> >> >> process, are when import numpy in python?

> >>

> >> >>

> >>

> >> >> if its the latter, you need to add a .conf file in  
> /etc/ld.so.conf.d/

> >>

> >> >> with the line /usr/local/rich/src/scipy_build/lib and then run sudo

> >>

> >> >> ldconfig

> >>

> >> >>

> >>

> >> >> Chris

> >>

> >> >>

> >>

> >> >>

> >>

> >> >> On Sat, Jun 6, 2009 at 1:42 PM, Chris Colbertsccolbert at gmail.com>

> >> >> wrote:

> >>

> >> >> > can you run this and post the build.log to pastebin.com:

> >>

> >> >> >

> >>

> >> >> > assuming your numpy build directory is /home/numpy-1.3.0:

> >>

> >> >> >

> >>

> >> >> > cd /home/numpy-1.3.0

> >>

> >> >> > rm -rf build

> >>

> >> >> > python setup.py build &&> build.log

> >>

> >> >> >

> >>

> >> >> >

> >>

> >> >> > Chris

> >>

> >> >> >

> >>

> >> >> >

> >>

> >> >> > On Sat, Jun 6, 2009 at 1:37 PM, Richard  
> Llewellynllewelr at gmail.com>

> >>

> >> >> > wrote:

> >>

> >> >> >> Hi Chris,

> >>

> >> >> >> thanks much for posting those installation instructions. Seems

> >>

> >> >> >> similar to

> >>

> >> >> >> what I pieced together.

> >>

> >> >> >>

> >>

> >> >> >> I gather ATLAS not found. Oops, drank that beer too early.

> >>

> >> >> >>

> >>

> >> >> >> I copied Atlas libs to /usr/local/rich/src/scipy_build/lib.

> >>

> >> >> >>

> >>

> >> >> >> This is my site.cfg. Out of desperation I tried  
> search_static_first

> >> >> >> =

> >>

> >> >> >> 1,

> >>

> >> >> >> but probably of no use.

> >>

> >> >> >>

> >>

> >> >> >> [DEFAULT]

> >>

> >> >> >> library_dirs =

> >> >> >> /usr/local/rich/src/scipy_build/lib:$HOME/usr/galois/lib

> >>

> >> >> >> include_dirs =

> >>

> >> >> >>  
> /usr/local/rich/src/scipy_build/lib/include:$HOME/usr/galois/include

> >>

> >> >> >> search_static_first = 1

> >>

> >> >> >>

> >>

> >> >> >> [blas_opt]

> >>

> >> >> >> libraries = f77blas, cblas, atlas

> >>

> >> >> >>

> >>

> >> >> >> [lapack_opt]

> >>

> >> >> >> libraries = lapack, f77blas, cblas, atlas

> >>

> >> >> >>

> >>

> >> >> >> [amd]

> >>

> >> >> >> amd_libs = amd

> >>

> >> >> >>

> >>

> >> >> >> [umfpack]

> >>

> >> >> >> umfpack_libs = umfpack, gfortran

> >>

> >> >> >>

> >>

> >> >> >> [fftw]

> >>

> >> >> >> libraries = fftw3

> >>

> >> >> >>

> >>

> >> >> >>

> >>

> >> >> >> Rich

> >>

> >> >> >>

> >>

> >> >> >>

> >>

> >> >> >>

> >>

> >> >> >>

> >>

> >> >> >> On Sat, Jun 6, 2009 at 10:25 AM, Chris Colbert  
> sccolbert at gmail.com>

> >>

> >> >> >> wrote:

> >>

> >> >> >>>

> >>

> >> >> >>> when you build numpy, did you use site.cfg to tell it where to  
> find

> >>

> >> >> >>> your atlas libs?

> >>

> >> >> >>>

> >>

> >> >> >>> On Sat, Jun 6, 2009 at 1:02 PM, Richard  
> Llewellynllewelr at gmail.com>

> >>

> >> >> >>> wrote:

> >>

> >> >> >>> > Hello,

> >>

> >> >> >>> >

> >>

> >> >> >>> > I've managed a build of lapack and atlas on Fedora 10 on a  
> quad

> >>

> >> >> >>> > core,

> >>

> >> >> >>> > 64,

> >>

> >> >> >>> > and now (...) have a numpy I can import that runs tests ok. :]

> >> >> >>> > I

> >>

> >> >> >>> > am

> >>

> >> >> >>> > puzzled, however, that numpy builds and imports lapack_lite.

> >> >> >>> > Does

> >>

> >> >> >>> > this

> >>

> >> >> >>> > mean

> >>

> >> >> >>> > I have a problem with the build(s)?

> >>

> >> >> >>> > Upon building numpy, I see the troubling output:

> >>

> >> >> >>> >

> >>

> >> >> >>> > ########################

> >>

> >> >> >>> >

> >>

> >> >> >>> > C compiler: gcc -pthread -fno-strict-aliasing -DNDEBUG -O2 -g

> >> >> >>> > -pipe

> >>

> >> >> >>> > -Wall

> >>

> >> >> >>> > -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protecto

> >>

> >> >> >>> > r --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE

> >> >> >>> > -fPIC

> >>

> >> >> >>> > -fPIC

> >>

> >> >> >>> >

> >>

> >> >> >>> > compile options: '-c'

> >>

> >> >> >>> > gcc: _configtest.c

> >>

> >> >> >>> > gcc -pthread _configtest.o  
> -L/usr/local/rich/src/scipy_build/lib

> >>

> >> >> >>> > -llapack

> >>

> >> >> >>> > -lptf77blas -lptcblas -latlas -o _configtest

> >>

> >> >> >>> > /usr/bin/ld: _configtest: hidden symbol `__powidf2' in

> >>

> >> >> >>> > /usr/lib/gcc/x86_64-redhat-linux/4.3.2/libgcc.a(_powidf2.o) is

> >>

> >> >> >>> > reference

> >>

> >> >> >>> > d by DSO

> >>

> >> >> >>> > /usr/bin/ld: final link failed: Nonrepresentable section on

> >> >> >>> > output

> >>

> >> >> >>> > collect2: ld returned 1 exit status

> >>

> >> >> >>> > /usr/bin/ld: _configtest: hidden symbol `__powidf2' in

> >>

> >> >> >>> > /usr/lib/gcc/x86_64-redhat-linux/4.3.2/libgcc.a(_powidf2.o) is

> >>

> >> >> >>> > reference

> >>

> >> >> >>> > d by DSO

> >>

> >> >> >>> > /usr/bin/ld: final link failed: Nonrepresentable section on

> >> >> >>> > output

> >>

> >> >> >>> > collect2: ld returned 1 exit status

> >>

> >> >> >>> > failure.

> >>

> >> >> >>> > removing: _configtest.c _configtest.o

> >>

> >> >> >>> > Status: 255

> >>

> >> >> >>> > Output:

> >>

> >> >> >>> > FOUND:

> >>

> >> >> >>> > libraries = ['lapack', 'ptf77blas', 'ptcblas', 'atlas']

> >>

> >> >> >>> > library_dirs = ['/usr/local/rich/src/scipy_build/lib']

> >>

> >> >> >>> > language = f77

> >>

> >> >> >>> > define_macros = [('NO_ATLAS_INFO', 2)]

> >>

> >> >> >>> >

> >>

> >> >> >>> > ##########################

> >>

> >> >> >>> >

> >>

> >> >> >>> > I don't have root on this machine, but could pester admins for

> >>

> >> >> >>> > eventual

> >>

> >> >> >>> > temporary access.

> >>

> >> >> >>> >

> >>

> >> >> >>> > Thanks much for any help,

> >>

> >> >> >>> > Rich

> >>

> >> >> >>> >

> >>

> >> >> >>> > _______________________________________________

> >>

> >> >> >>> > Numpy-discussion mailing list

> >>

> >> >> >>> > Numpy-discussion at scipy.org

> >>

> >> >> >>> > http://mail.scipy.org/mailman/listinfo/numpy-discussion

> >>

> >> >> >>> >

> >>

> >> >> >>> >

> >>

> >> >> >>> _______________________________________________

> >>

> >> >> >>> Numpy-discussion mailing list

> >>

> >> >> >>> Numpy-discussion at scipy.org

> >>

> >> >> >>> http://mail.scipy.org/mailman/listinfo/numpy-discussion

> >>

> >> >> >>

> >>

> >> >> >>

> >>

> >> >> >> _______________________________________________

> >>

> >> >> >> Numpy-discussion mailing list

> >>

> >> >> >> Numpy-discussion at scipy.org

> >>

> >> >> >> http://mail.scipy.org/mailman/listinfo/numpy-discussion

> >>

> >> >> >>

> >>

> >> >> >>

> >>

> >> >> >

> >>

> >> >> _______________________________________________

> >>

> >> >> Numpy-discussion mailing list

> >>

> >> >> Numpy-discussion at scipy.org

> >>

> >> >> http://mail.scipy.org/mailman/listinfo/numpy-discussion

> >>
> %
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20090607/ad5194ac/attachment.html>