Hi! I have been using numpy+scipy+matplotlib extensively in scripts since quite some time, and regularly updated all these packages using svn. For some reason the following does not work anymore, and is trying to find cblas_dtrsm. I have it in /usr/local/lib/atlas/libcblas.a (I checked doing a "ar -t" of that library). I also have that directory in my ld.so.conf (and indeed as ATLAS default dir). it must be a very stupid mistake I am making here, but I cannot make it work. (it always worked, and I haven't changed anything in my ATLAS/BLAS, etc) (I tried recompiling scipy but no luck) Anyone has an idea of what is wrong with that? thanks for any help here. Eric ==================================== ---> 21 from scipy import optimize, special, interpolate 22 from scipy.optimize import leastsq, fsolve 23 from scipy.special import gammainc /usr/local/lib/python2.4/site-packages/scipy/optimize/__init__.py 9 from zeros import * 10 from anneal import * ---> 11 from lbfgsb import fmin_l_bfgs_b 12 from tnc import fmin_tnc 13 from cobyla import fmin_cobyla /usr/local/lib/python2.4/site-packages/scipy/optimize/lbfgsb.py 28 29 from numpy import zeros, float64, array, int32 ---> 30 import _lbfgsb 31 import optimize 32 ImportError: /usr/local/lib/python2.4/site-packages/scipy/optimize/_lbfgsb.so: undefined symbol: cblas_dtrsm
Eric Emsellem wrote:
Hi!
I have been using numpy+scipy+matplotlib extensively in scripts since quite some time, and regularly updated all these packages using svn.
For some reason the following does not work anymore, and is trying to find cblas_dtrsm. I have it in /usr/local/lib/atlas/libcblas.a (I checked doing a "ar -t" of that library). I also have that directory in my ld.so.conf (and indeed as ATLAS default dir).
it must be a very stupid mistake I am making here, but I cannot make it work. (it always worked, and I haven't changed anything in my ATLAS/BLAS, etc)
(I tried recompiling scipy but no luck)
Anyone has an idea of what is wrong with that?
Not without knowing the details of how you built scipy. Can you show us the output of $ python setup.py config ? Also, I believe that you can find symbols which are statically linked into the extension module by using nm(1) (but I'm too lazy to find something that I can check; I use a shared-library ATLAS on my Linux system). $ nm $PYLIB/scipy/optimize/_lbfgsb.so | grep cblas_dtrsm -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Eric Emsellem wrote:
thanks for the quick reply.
doing the nm I get: U cblas_dtrsm
and the python config: (not much seems to be found?)
Okay, then you should make a site.cfg file with the information to find and link against your ATLAS libraries. If you have a relatively recent numpy, there should be a site.cfg.example file with plenty of comments on how to fill it out. Otherwise, you can get that file from here: http://svn.scipy.org/svn/numpy/trunk/site.cfg.example -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Eric Emsellem wrote:
HI again
well I have now spent a lot of time trying to get back to my normal setup for scipy and nothing seems to work. I still get an error message on cblas_dtrsm even though I have tried to include all libraries I can think of, to rewrite the links to these even using the site.cfg (it always worked before without that but I followed your advice there).
Okay, what are the contents of your site.cfg? -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Well the uncommented lines are: # [DEFAULT] library_dirs = /usr/local/lib include_dirs = /usr/local/include # [blas_opt] libraries = f77blas, cblas, atlas # [lapack_opt] libraries = lapack, f77blas, cblas, atlas # [fftw] libraries = fftw3 it finds the libraries but not the _info part (not sure this is critical..?). Eric -- ==================================================================== Eric Emsellem emsellem@obs.univ-lyon1.fr Centre de Recherche Astrophysique de Lyon 9 av. Charles-Andre tel: +33 (0)4 78 86 83 84 69561 Saint-Genis Laval Cedex fax: +33 (0)4 78 86 83 86 France http://www-obs.univ-lyon1.fr/eric.emsellem ====================================================================
Eric Emsellem wrote:
Well the uncommented lines are:
# [DEFAULT] library_dirs = /usr/local/lib include_dirs = /usr/local/include # [blas_opt] libraries = f77blas, cblas, atlas # [lapack_opt] libraries = lapack, f77blas, cblas, atlas
These libraries are all in /usr/local/lib/atlas, right? So add library_dirs = /usr/local/lib/atlas to [blas_opt] and [lapack_opt]
# [fftw] libraries = fftw3
it finds the libraries but not the _info part (not sure this is critical..?).
No, the _info part comes the names of the classes in the code that implement each section. Nothing to worry about. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Hi, after the trouble I had with lapack and blas, I decided to reinstall everything.. Now I end up with a different (still annoying) message about libraries not being found. Here it is (and below I provide the details of what I have done to install everything again). Any help is VERY welcome ....!!! Eric ### Doing a config on scipy gives: python setup.py config Traceback (most recent call last): File "setup.py", line 55, in ? setup_package() File "setup.py", line 28, in setup_package from numpy.distutils.core import setup File "/usr/local/lib/python2.4/site-packages/numpy/__init__.py", line 40, in ? import linalg File "/usr/local/lib/python2.4/site-packages/numpy/linalg/__init__.py", line 4, in ? from linalg import * File "/usr/local/lib/python2.4/site-packages/numpy/linalg/linalg.py", line 25, in ? from numpy.linalg import lapack_lite ImportError: /usr/local/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so: undefined symbol: ATL_cGetNB ######### Here are the details of my installation: ################# NUMPY/SCIPY ################## ## INSTALLATION including blas, lapack, atlas ## ################################################ ## Getting the svn for numpy and scipy svn co http://svn.scipy.org/svn/numpy/trunk numpy svn co http://svn.scipy.org/svn/scipy/trunk scipy ## allowing root su ## build and install numpy cd numpy python setup.py install >& inst.log & tail -f inst.log ## Going to scipy directory to build everything else cd .. cd scipy ## First BLAS mkdir -p blas cd blas ## Getting the archive wget http://www.netlib.org/blas/blas.tgz tar xvzf blas.tgz ## Compiling g77 -fno-second-underscore -O2 -c *.f ## Making the library ar r libfblas.a *.o ranlib libfblas.a ## Cleaning rm -rf *.o ## Coping the output where it should be cp libfblas.a /usr/local/lib # For bash use export, for tcsh use setenv of course # export BLAS=/usr/local/lib/libfblas.a setenv BLAS /usr/local/lib/libfblas.a ## Doing LAPACK now cd .. wget http://www.netlib.org/lapack/lapack.tgz tar xzf lapack.tgz cd lapack-3.1.0 cp INSTALL/make.inc.LINUX make.inc ###### Now you must edit make.inc and change (if necessary) the following values: OPTS = "-O2" ###### ## !!!!!! ## WARNING ## With 10.2 OpenSuse you must do: ## cd /lib ## ln -s libgcc_s.so.1 libgcc_s.so ## Maybe it will be corrected later ## !!!!!! ###### make lapacklib >& make.log & tail -f make.log ###### ## Now we obtain and compile the ATLAS package. Get it from: ## https://sourceforge.net/projects/math-atlas/ ###### ## ## !!!!!! ## Seems that version 3.6.0 does not always work and provide errors such as: ## make[3]: *** [res/cMMRES] Erreur 255 ## GetMMRES: Assertion `fp' failed. ## ## -> so getting the latest "unstable version": 3.7.25 ## cd scipy bunzip2 atlas3.7.25.tar.bz2 tar xvf atlas3.7.25.tar ## CPU THROTTLING OFF (otherwise ATLAS builing makes no sense ) cpufreq-set -g performance ## compiling ATLAS cd ATLAS ## giving a name for this build mkdir ATLAS_Linux_P4E ; cd ATLAS_Linux_P4E ../configure ## Now merging the lapack lib and the programs from atlas make cd lib mkdir tmp cd tmp ## extracting the obj from ATLAS lapack lib ar x ../liblapack.a cp ../liblapack.a ../liblapack.a.sav cp ../../../../lapack-3.1.0/lapack_LINUX.a ../liblapack.a ## adding files to the lapack lib from lapack ar r ../liblapack.a *.o cd .. rm -rf tmp cp liblapack.a /usr/local/lib # export LAPACK=/usr/local/lib/liblapack.a setenv LAPACK /usr/local/lib/liblapack.a mkdir /usr/local/lib/atlas cp *.a /usr/local/lib/atlas # export ATLAS=/usr/local/lib/atlas setenv ATLAS /usr/local/lib/atlas ## Until that point everything seems to have gone ok, but now... cd ../../.. python setup.py install ==> crashing with the message mentioned above -- ==================================================================== Eric Emsellem emsellem@obs.univ-lyon1.fr Centre de Recherche Astrophysique de Lyon 9 av. Charles-Andre tel: +33 (0)4 78 86 83 84 69561 Saint-Genis Laval Cedex fax: +33 (0)4 78 86 83 86 France http://www-obs.univ-lyon1.fr/eric.emsellem ====================================================================
Hi, undefined symbol: ATL_cGetNB : Ok, looks like you have problem with the way you link lapack, blass and atlas after you compile it. I cannot remember your way to compile all this stuff because it is not needed on my box :). Google tells me that "libatlas needs to be linked in after liblapack" A line containing //"-latlas -llapack" is wrong. It should be "//-llapack -latlas" or "-llapack -lf77blas -latlas" if needed. I hope this helps. I have no clue why you haven't seen this bug before. Maybe just because this function was not yet wrapped in scipy... Xavier
Hi,
after the trouble I had with lapack and blas, I decided to reinstall everything.. Now I end up with a different (still annoying) message about libraries not being found. Here it is (and below I provide the details of what I have done to install everything again).
Any help is VERY welcome ....!!!
Eric
### Doing a config on scipy gives: python setup.py config
Traceback (most recent call last): File "setup.py", line 55, in ? setup_package() File "setup.py", line 28, in setup_package from numpy.distutils.core import setup File "/usr/local/lib/python2.4/site-packages/numpy/__init__.py", line 40, in ? import linalg File "/usr/local/lib/python2.4/site-packages/numpy/linalg/__init__.py", line 4, in ? from linalg import * File "/usr/local/lib/python2.4/site-packages/numpy/linalg/linalg.py", line 25, in ? from numpy.linalg import lapack_lite ImportError: /usr/local/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so: undefined symbol: ATL_cGetNB
######### Here are the details of my installation: ################# NUMPY/SCIPY ################## ## INSTALLATION including blas, lapack, atlas ## ################################################
## Getting the svn for numpy and scipy svn co http://svn.scipy.org/svn/numpy/trunk numpy svn co http://svn.scipy.org/svn/scipy/trunk scipy
## allowing root su
## build and install numpy cd numpy python setup.py install >& inst.log & tail -f inst.log
## Going to scipy directory to build everything else cd .. cd scipy
## First BLAS mkdir -p blas cd blas ## Getting the archive wget http://www.netlib.org/blas/blas.tgz tar xvzf blas.tgz ## Compiling g77 -fno-second-underscore -O2 -c *.f ## Making the library ar r libfblas.a *.o ranlib libfblas.a ## Cleaning rm -rf *.o ## Coping the output where it should be cp libfblas.a /usr/local/lib # For bash use export, for tcsh use setenv of course # export BLAS=/usr/local/lib/libfblas.a setenv BLAS /usr/local/lib/libfblas.a
## Doing LAPACK now cd .. wget http://www.netlib.org/lapack/lapack.tgz tar xzf lapack.tgz cd lapack-3.1.0 cp INSTALL/make.inc.LINUX make.inc
###### Now you must edit make.inc and change (if necessary) the following values: OPTS = "-O2" ###### ## !!!!!! ## WARNING ## With 10.2 OpenSuse you must do: ## cd /lib ## ln -s libgcc_s.so.1 libgcc_s.so ## Maybe it will be corrected later ## !!!!!! ######
make lapacklib >& make.log & tail -f make.log
###### ## Now we obtain and compile the ATLAS package. Get it from: ## https://sourceforge.net/projects/math-atlas/ ###### ## ## !!!!!! ## Seems that version 3.6.0 does not always work and provide errors such as: ## make[3]: *** [res/cMMRES] Erreur 255 ## GetMMRES: Assertion `fp' failed. ## ## -> so getting the latest "unstable version": 3.7.25 ## cd scipy bunzip2 atlas3.7.25.tar.bz2 tar xvf atlas3.7.25.tar
## CPU THROTTLING OFF (otherwise ATLAS builing makes no sense ) cpufreq-set -g performance ## compiling ATLAS cd ATLAS ## giving a name for this build mkdir ATLAS_Linux_P4E ; cd ATLAS_Linux_P4E ../configure
## Now merging the lapack lib and the programs from atlas make cd lib mkdir tmp cd tmp ## extracting the obj from ATLAS lapack lib ar x ../liblapack.a cp ../liblapack.a ../liblapack.a.sav cp ../../../../lapack-3.1.0/lapack_LINUX.a ../liblapack.a ## adding files to the lapack lib from lapack ar r ../liblapack.a *.o cd .. rm -rf tmp cp liblapack.a /usr/local/lib # export LAPACK=/usr/local/lib/liblapack.a setenv LAPACK /usr/local/lib/liblapack.a mkdir /usr/local/lib/atlas cp *.a /usr/local/lib/atlas # export ATLAS=/usr/local/lib/atlas setenv ATLAS /usr/local/lib/atlas
## Until that point everything seems to have gone ok, but now... cd ../../.. python setup.py install
==> crashing with the message mentioned above
[Pre. S.: You don't have to include me on the To: line as well. I'm on the list, and it messes up my mail filters (which admittedly are probably pretty lame).] Eric Emsellem wrote:
Hi,
after the trouble I had with lapack and blas, I decided to reinstall everything.. Now I end up with a different (still annoying) message about libraries not being found. Here it is (and below I provide the details of what I have done to install everything again).
Any help is VERY welcome ....!!!
Eric
### Doing a config on scipy gives: python setup.py config
Traceback (most recent call last): File "setup.py", line 55, in ? setup_package() File "setup.py", line 28, in setup_package from numpy.distutils.core import setup File "/usr/local/lib/python2.4/site-packages/numpy/__init__.py", line 40, in ? import linalg File "/usr/local/lib/python2.4/site-packages/numpy/linalg/__init__.py", line 4, in ? from linalg import * File "/usr/local/lib/python2.4/site-packages/numpy/linalg/linalg.py", line 25, in ? from numpy.linalg import lapack_lite ImportError: /usr/local/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so: undefined symbol: ATL_cGetNB
Xavier is probably correct in his assessment why you get this error. However, it looks like you built ATLAS after numpy which is odd. Make sure that you build ATLAS first. You're putting it in /usr/local/lib/atlas/ which is fine. Now, make a site.cfg file with the right information. # The order of the libraries is important! Don't change them! [blas_opt] library_dirs = /usr/local/lib/atlas/ libraries = f77blas, cblas, atlas [lapack_opt] library_dirs = /usr/local/lib/atlas/ libraries = lapack, f77blas, cblas, atlas Now build numpy. Put the site.cfg file next to the setup.py file in the root of your checkout directory. The install should go smoothly. Run the test suite to make sure. $ python -c "import numpy; numpy.test()" Now build scipy. Again, put the site.cfg file next to the setup.py file in the root of your checkout directory. The install should also go smoothly. Run the test suite to make sure. $ python -c "import scipy; scipy.test()" -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Hi, after reinstalling EVERYTHING (atlas, lapack, etc) I still hit the wall of "no way to install scipy". It seems that it is still a problem with linking (and maybe at the numpy stage in fact). When doing python setup.py config, for testing the config of scipy, I get an undefined symbol in Traceback (most recent call last): File "setup.py", line 55, in ? setup_package() ............ ImportError: /usr/local/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so: undefined symbol: ATL_cGetNB The ATL_cGetNB.o is in fact included in the libatlas.a, so I wonder why it did not find it. The numpy installation went well in fact (at least apparently). Any clue here? thanks a lot Eric P.S.: this is a desperate shot.. I have been trying to install scipy for 2 weeks now. -- ==================================================================== Eric Emsellem emsellem@obs.univ-lyon1.fr Centre de Recherche Astrophysique de Lyon 9 av. Charles-Andre tel: +33 (0)4 78 86 83 84 69561 Saint-Genis Laval Cedex fax: +33 (0)4 78 86 83 86 France http://www-obs.univ-lyon1.fr/eric.emsellem ====================================================================
Eric Emsellem wrote:
Any clue here?
Not until you show us your site.cfg that you used to build numpy. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
participants (3)
-
Eric Emsellem -
Gnata Xavier -
Robert Kern