Numpy doesn't seem to be finding my atlas install. Have I done something wrong or misunderstood? $ cd /usr/lib $ ls libatlas* libatlas.a libatlas.so libatlas.so.3gf libatlas.so.3gf.0 $ ls libf77* libf77blas.a libf77blas.so libf77blas.so.3gf libf77blas.so.3gf.0 $ ls libcblas* libcblas.a libcblas.so libcblas.so.3gf libcblas.so.3gf.0 $ ls liblapack* liblapack-3.a liblapack.a liblapack_atlas.so liblapack_atlas.so.3gf.0 liblapackgf-3.so liblapack.so.3gf liblapack-3.so liblapack_atlas.a liblapack_atlas.so.3gf liblapackgf-3.a liblapack.so liblapack.so.3gf.0 Since these are all in the standard locations, I am building without a site.cfg. Here is the beginning info: Running from numpy source directory. non-existing path in 'numpy/distutils': 'site.cfg' F2PY Version 2_5972 blas_opt_info: blas_mkl_info: libraries mkl,vml,guide not found in /usr/local/lib libraries mkl,vml,guide not found in /usr/lib NOT AVAILABLE atlas_blas_threads_info: Setting PTATLAS=ATLAS NOT AVAILABLE atlas_blas_info: NOT AVAILABLE /tmp/numpy/numpy/distutils/system_info.py:1340: UserWarning: Atlas (http://math-atlas.sourceforge.net/) libraries not found. Directories to search for the libraries can be specified in the numpy/distutils/site.cfg file (section [atlas]) or by setting the ATLAS environment variable. warnings.warn(AtlasNotFoundError.__doc__) blas_info: libraries blas not found in /usr/local/lib FOUND: libraries = ['blas'] library_dirs = ['/usr/lib'] language = f77 FOUND: libraries = ['blas'] library_dirs = ['/usr/lib'] define_macros = [('NO_ATLAS_INFO', 1)] language = f77 lapack_opt_info: lapack_mkl_info: mkl_info: libraries mkl,vml,guide not found in /usr/local/lib libraries mkl,vml,guide not found in /usr/lib NOT AVAILABLE NOT AVAILABLE atlas_threads_info: Setting PTATLAS=ATLAS numpy.distutils.system_info.atlas_threads_info NOT AVAILABLE atlas_info: numpy.distutils.system_info.atlas_info NOT AVAILABLE /tmp/numpy/numpy/distutils/system_info.py:1247: UserWarning: Atlas (http://math-atlas.sourceforge.net/) libraries not found. Directories to search for the libraries can be specified in the numpy/distutils/site.cfg file (section [atlas]) or by setting the ATLAS environment variable. warnings.warn(AtlasNotFoundError.__doc__) lapack_info: libraries lapack not found in /usr/local/lib FOUND: libraries = ['lapack'] library_dirs = ['/usr/lib'] language = f77 FOUND: libraries = ['lapack', 'blas'] library_dirs = ['/usr/lib'] define_macros = [('NO_ATLAS_INFO', 1)] language = f77
On Mon, Nov 3, 2008 at 10:46 AM, T J <tjhnson@gmail.com> wrote:
Since these are all in the standard locations, I am building without a site.cfg. Here is the beginning info:
Apparently, this is not enough. Only if I also set the ATLAS environment variable am I able to get this working as expected. So while I now have this working, I'd still like to understand why this is necessary. This is for Ubuntu 8.10. My previous experience was that no site.cfg was necessary. This is still the case, but you also need ATLAS defined. I was also expecting that the following site.cfg would avoid requiring ATLAS to be defined, but it did not. [DEFAULT] library_dirs = /usr/lib:/usr/lib/sse2 include_dirs = /usr/local/include [blas_opt] libraries = f77blas, cblas, atlas [lapack_opt] libraries = lapack, f77blas, cblas, atlas So can someone explain why I *must* define ATLAS. I tried a number of variations on site.cfg and could not get numpy to find atlas with any of them.
On Tue, Nov 4, 2008 at 7:59 AM, T J <tjhnson@gmail.com> wrote:
So can someone explain why I *must* define ATLAS. I tried a number of variations on site.cfg and could not get numpy to find atlas with any of them.
Ok, I took a brief look at this: I forgot that Ubuntu and Debian added an aditional library suffix to libraries depending on gfortran ABI. I added support for this in numpy.distutils - which was looking for libraries explicitely; could you retry *without* a site.cfg ? It should work, now, David
David Cournapeau wrote:
Ok, I took a brief look at this: I forgot that Ubuntu and Debian added an aditional library suffix to libraries depending on gfortran ABI. I added support for this in numpy.distutils - which was looking for libraries explicitely; could you retry *without* a site.cfg ? It should work, now,
And it won't, because I am afraid the Ubuntu atlas package is broken in 8.10... They use the gfortran ABI, but they built the fortran wrappers with g77, according to ATL_buildinfo. https://bugs.launchpad.net/ubuntu/+source/atlas/+bug/295051 I would strongly advise using atlas on Ubuntu 8.10 until this bug is solved: it means any numpy/scipy code using linear algebra is potentially broken (segfault, wrong results). cheers, David
On Fri, Nov 7, 2008 at 1:26 AM, David Cournapeau <david@ar.media.kyoto-u.ac.jp> wrote:
David Cournapeau wrote:
Ok, I took a brief look at this: I forgot that Ubuntu and Debian added an aditional library suffix to libraries depending on gfortran ABI. I added support for this in numpy.distutils - which was looking for libraries explicitly; could you retry *without* a site.cfg ? It should work, now,
And it won't, because I am afraid the Ubuntu atlas package is broken in 8.10... They use the gfortran ABI, but they built the fortran wrappers with g77, according to ATL_buildinfo.
https://bugs.launchpad.net/ubuntu/+source/atlas/+bug/295051
I would strongly advise using atlas on Ubuntu 8.10 until this bug is solved: it means any numpy/scipy code using linear algebra is potentially broken (segfault, wrong results).
Intended: '''I would strongly advise *against* using atlas on Ubuntu 8.10.''' :-) That the fortran wrappers were compiled using g77 is also apparent via what is printed out during setup when ATLAS is detected: gcc -pthread _configtest.o -L/usr/lib/atlas -llapack -lblas -o _configtest ATLAS version 3.6.0 built by root on Fri Jan 9 15:57:20 UTC 2004: UNAME : Linux intech67 2.4.20 #1 SMP Fri Jan 10 18:29:51 EST 2003 i686 GNU/Linux INSTFLG : MMDEF : /fix/g/camm/atlas3-3.6.0/CONFIG/ARCHS/P4SSE2/gcc/gemm ARCHDEF : /fix/g/camm/atlas3-3.6.0/CONFIG/ARCHS/P4SSE2/gcc/misc F2CDEFS : -DAdd__ -DStringSunStyle CACHEEDGE: 1048576 F77 : /usr/bin/g77, version GNU Fortran (GCC) 3.3.3 20031229 (prerelease) (Debian) F77FLAGS : -fomit-frame-pointer -O CC : /usr/bin/gcc, version gcc (GCC) 3.3.3 20031229 (prerelease) (Debian) CC FLAGS : -fomit-frame-pointer -O3 -funroll-all-loops MCC : /usr/bin/gcc, version gcc (GCC) 3.3.3 20031229 (prerelease) (Debian) MCCFLAGS : -fomit-frame-pointer -O success! And the problem with this, as you've mentioned before, is that g77 and gfortran are not abi compatible. But isn't this issue separate from the autodetection of atlas without a site.cfg? With r5986, atlas is still only detected if I declare ATLAS: $ ATLAS=/usr/lib python setup.py build versus $ unset ATLAS; python setup.py build
On Fri, Nov 7, 2008 at 1:48 AM, David Cournapeau <david@ar.media.kyoto-u.ac.jp> wrote:
It works for me on Intrepid (64 bits). Did you install libatlas3gf-base-dev ? (the names changed in intrepid).
I fear I am overlooking something obvious. $ sudo aptitude search libatlas p libatlas-3dnow-dev - Automatically Tuned Linear Algebra Software,3dnow static v libatlas-3gf.so - i libatlas-base-dev - Automatically Tuned Linear Algebra Software,generic static p libatlas-cpp-0.6-1 - The protocol library of the World Forge project - runtime libs p libatlas-cpp-0.6-1-dbg - The protocol library of the World Forge project - debugging libs p libatlas-cpp-0.6-dev - The protocol library of the World Forge project - header files p libatlas-cpp-doc - The protocol library of the World Forge project - documentation p libatlas-doc - Automatically Tuned Linear Algebra Software,documentation i A libatlas-headers - Automatically Tuned Linear Algebra Software,C header files p libatlas-sse-dev - Automatically Tuned Linear Algebra Software,SSE1 static i libatlas-sse2-dev - Automatically Tuned Linear Algebra Software,SSE2 static p libatlas-test - Automatically Tuned Linear Algebra Software,test programs v libatlas.so.3 - v libatlas.so.3gf - p libatlas3gf-3dnow - Automatically Tuned Linear Algebra Software,3dnow shared i A libatlas3gf-base - Automatically Tuned Linear Algebra Software,generic shared p libatlas3gf-sse - Automatically Tuned Linear Algebra Software,SSE1 shared i libatlas3gf-sse2 - Automatically Tuned Linear Algebra Software,SSE2 shared It looks like I have the important ones: libatlas-base-dev libatlas-headers libatlas-sse2-dev libatlas3gf-base libatlas3gf-sse2 But I don't see libatlas3gf-base-dev anywhere. Is this on a special repository? My sources.list is: # Intrepid Final Release Repository deb http://archive.ubuntu.com/ubuntu intrepid main restricted universe multiverse deb-src http://archive.ubuntu.com/ubuntu intrepid main restricted universe multiverse #Added by software-properties # Intrepid Security Updates deb http://archive.ubuntu.com/ubuntu intrepid-security main restricted universe multiverse deb-src http://archive.ubuntu.com/ubuntu intrepid-security main restricted universe multiverse #Added by software-properties # Intrepid Bugfix Updates deb http://archive.ubuntu.com/ubuntu intrepid-updates main restricted universe multiverse deb-src http://archive.ubuntu.com/ubuntu intrepid-updates main restricted universe multiverse #Added by software-properties I suppose it is possible that something is messed up on my end since I upgraded from 8.04.
T J wrote:
I fear I am overlooking something obvious.
No, I mixed up the names, I meant libatlas-base-dev.
It looks like I have the important ones: libatlas-base-dev libatlas-headers libatlas-sse2-dev libatlas3gf-base libatlas3gf-sse2
I have only libatlas-headers, libatlas-base-dev and libatlas3gf-base installed. This is strange. You are building numpy in the top source tree, right ? And you have no site.cfg at all ? cheers, David
On Fri, Nov 7, 2008 at 2:16 AM, David Cournapeau <david@ar.media.kyoto-u.ac.jp> wrote:
And you have no site.cfg at all ?
Wow. I was too focused on the current directory and didn't realize I had an old site.cfg in ~/. Two points: 1) Others (myself included) might catch such silliness sooner if the location of the cfg file were printed to screen during setup.py. As of now, I get: Running from numpy source directory. non-existing path in 'numpy/distutils': 'site.cfg' F2PY Version 2_5986 ... 2) The current system_info.py says: """ The file 'site.cfg' is looked for in 1) Directory of main setup.py file being run. 2) Home directory of user running the setup.py file as ~/.numpy-site.cfg 3) System wide directory (location of this file...) """ Doesn't this mean that it should *not* have picked up my ~/site.cfg? Anyway, I can now report that ATLAS is detected without defining any environment variables. Thanks for all the help!
On Fri, Nov 7, 2008 at 8:10 PM, T J <tjhnson@gmail.com> wrote:
On Fri, Nov 7, 2008 at 2:16 AM, David Cournapeau <david@ar.media.kyoto-u.ac.jp> wrote:
And you have no site.cfg at all ?
Wow. I was too focused on the current directory and didn't realize I had an old site.cfg in ~/.
Two points:
1) Others (myself included) might catch such silliness sooner if the location of the cfg file were printed to screen during setup.py. As of now, I get
I agree this is confusing. Our build process generally produces way too much output, and not always useful information. FOr the site.cfg, there are a lot of possibilities, I think the whole process is a bit too smart for its own good, but it is hard to change without breaking someone else workflow. I will take a look at the code to see if it is doable to print the actual site.cfg used if it is indeed used.
Running from numpy source directory. non-existing path in 'numpy/distutils': 'site.cfg' F2PY Version 2_5986 ...
2) The current system_info.py says:
""" The file 'site.cfg' is looked for in
1) Directory of main setup.py file being run. 2) Home directory of user running the setup.py file as ~/.numpy-site.cfg 3) System wide directory (location of this file...) """
Doesn't this mean that it should *not* have picked up my ~/site.cfg?
Yes, it seems. To be honest, I never used this feature, I am surprised myself it picked up $HOME/site.cfg.
Anyway, I can now report that ATLAS is detected without defining any environment variables.
This the fortran wrappers of the ATLAS are broken, I am not sure it is such a great news :) David
On Fri, Nov 7, 2008 at 1:58 AM, T J <tjhnson@gmail.com> wrote:
That the fortran wrappers were compiled using g77 is also apparent via what is printed out during setup when ATLAS is detected:
gcc -pthread _configtest.o -L/usr/lib/atlas -llapack -lblas -o _configtest ATLAS version 3.6.0 built by root on Fri Jan 9 15:57:20 UTC 2004: UNAME : Linux intech67 2.4.20 #1 SMP Fri Jan 10 18:29:51 EST 2003 i686 GNU/Linux INSTFLG : MMDEF : /fix/g/camm/atlas3-3.6.0/CONFIG/ARCHS/P4SSE2/gcc/gemm ARCHDEF : /fix/g/camm/atlas3-3.6.0/CONFIG/ARCHS/P4SSE2/gcc/misc F2CDEFS : -DAdd__ -DStringSunStyle CACHEEDGE: 1048576 F77 : /usr/bin/g77, version GNU Fortran (GCC) 3.3.3 20031229 (prerelease) (Debian) F77FLAGS : -fomit-frame-pointer -O CC : /usr/bin/gcc, version gcc (GCC) 3.3.3 20031229 (prerelease) (Debian) CC FLAGS : -fomit-frame-pointer -O3 -funroll-all-loops MCC : /usr/bin/gcc, version gcc (GCC) 3.3.3 20031229 (prerelease) (Debian) MCCFLAGS : -fomit-frame-pointer -O success!
That was intended to be a question.
T J wrote:
On Fri, Nov 7, 2008 at 1:58 AM, T J <tjhnson@gmail.com> wrote:
That the fortran wrappers were compiled using g77 is also apparent via what is printed out during setup when ATLAS is detected:
gcc -pthread _configtest.o -L/usr/lib/atlas -llapack -lblas -o _configtest ATLAS version 3.6.0 built by root on Fri Jan 9 15:57:20 UTC 2004: UNAME : Linux intech67 2.4.20 #1 SMP Fri Jan 10 18:29:51 EST 2003 i686 GNU/Linux INSTFLG : MMDEF : /fix/g/camm/atlas3-3.6.0/CONFIG/ARCHS/P4SSE2/gcc/gemm ARCHDEF : /fix/g/camm/atlas3-3.6.0/CONFIG/ARCHS/P4SSE2/gcc/misc F2CDEFS : -DAdd__ -DStringSunStyle CACHEEDGE: 1048576 F77 : /usr/bin/g77, version GNU Fortran (GCC) 3.3.3 20031229 (prerelease) (Debian) F77FLAGS : -fomit-frame-pointer -O CC : /usr/bin/gcc, version gcc (GCC) 3.3.3 20031229 (prerelease) (Debian) CC FLAGS : -fomit-frame-pointer -O3 -funroll-all-loops MCC : /usr/bin/gcc, version gcc (GCC) 3.3.3 20031229 (prerelease) (Debian) MCCFLAGS : -fomit-frame-pointer -O success!
That was intended to be a question.
Yes, this output is generated by a call to atlas function ATL_buildinfo, the exact same I used for the bug report, David
participants (3)
-
David Cournapeau -
David Cournapeau -
T J