I've installed atlas (atlas-3.2.1 and atlas-c++-0.4.5). "/usr/lib" is in my "/etc/ld.so.conf" and I've run "ldconfig". When I try installing scipy (cvs), I get the messages below. Can someone tell me what I'm doing wrong? (I did notice that there is a "/usr/bin/atlas-config" executable in the atlas distribution; I'm not sure how to use it though.) -- Stephen # python setup.py install atlas_info: NOT AVAILABLE linalg/setup_linalg.py:36: UserWarning: Atlas (http://math-atlas.sourceforge.net/) libraries not found. Directories to search for the libraries can be specified in the scipy_distutils/site.cfg file (section [atlas]) or by setting the ATLAS environment variable. warnings.warn(AtlasNotFoundError.__doc__) blas_info: NOT AVAILABLE linalg/setup_linalg.py:40: UserWarning: Blas (http://www.netlib.org/blas/) libraries not found. Directories to search for the libraries can be specified in the scipy_distutils/site.cfg file (section [blas]) or by setting the BLAS environment variable. warnings.warn(BlasNotFoundError.__doc__) blas_src_info: NOT AVAILABLE Traceback (most recent call last): File "setup.py", line 130, in ? install_package() File "setup.py", line 95, in install_package config.extend([get_package_config(x,parent_package)for x in standard_packages]) File "setup.py", line 45, in get_package_config config = mod.configuration(parent) File "linalg/setup_linalg.py", line 43, in configuration raise BlasSrcNotFoundError,BlasSrcNotFoundError.__doc__ scipy_distutils.system_info.BlasSrcNotFoundError: Blas (http://www.netlib.org/blas/) sources not found. Directories to search for the sources can be specified in the scipy_distutils/site.cfg file (section [blas_src]) or by setting the BLAS_SRC environment variable.
Stephen> I've installed atlas (atlas-3.2.1 and Stephen> atlas-c++-0.4.5). "/usr/lib" is in my "/etc/ld.so.conf" and Stephen> I've run "ldconfig". Stephen> When I try installing scipy (cvs), I get the messages Stephen> below. Can someone tell me what I'm doing wrong? I take it you installed atlas in /usr/lib? Did you set the ATLAS environment variable (I don't know if it looks in common places or if it always requires ATLAS to be set)? export ATLAS=/usr/lib If that doesn't work, check to make sure you've properly merged your atlas and lapack libraries. (Check the atlas FAQ for details on how to do this.) -- Skip Montanaro - skip@pobox.com http://www.mojam.com/ http://www.musi-cal.com/
On Sun, 13 Oct 2002, Skip Montanaro wrote:
Did you set the ATLAS environment variable (I don't know if it looks in common places or if it always requires ATLAS to be set)?
( scipy/system_info.py looks ATLAS libraries in all common places such as /usr/lib,/usr/local/lib,/opt/lib,/usr/lib/atlas,<sys.prefix>/lib, etc etc (but not necessary in the given order) and setting ATLAS environment variable is *not* an requirement. ATLAS environment variable can be useful only if 1) ATLAS is installed in a rather uncommon place, 2) or if one has different ATLAS libraries in the system and then ATLAS environment variable can be used to specify which one scipy should use. However, scipy/system_info.py refuses to find ATLAS libraries that are incomplete. The definition of completness is as follows: The following four files liblapack.a, libf77blas.a, libcblas.a, libatlas.a must exist in the *same* directory and the size of liblapack.a should be around 6MB (that is, the instructions in http://math-atlas.sourceforge.net/errata.html#completelp are succesfully applied). If any of these conditions is unsatisfied, expect problems when importing scipy. ) Pearu
On Mon, Oct 14, 2002 at 09:08:33PM +0300, Pearu Peterson wrote:
However, scipy/system_info.py refuses to find ATLAS libraries that are incomplete. The definition of completness is as follows:
The following four files
liblapack.a, libf77blas.a, libcblas.a, libatlas.a
Yeah, this is the problem in the case of a Debian system. liblapack.a is in /usr/lib/3dnow/atlas, and the rest of the libraries are in /usr/lib/3dnow. After doing a symbolic link from /usr/lib/3dnow/liblapack.a to /usr/lib/3dnow/atlas/liblapack.a, scipy detects the libraries and everything goes smoothly:
import scipy scipy.test(10) . <stuff skipped> . function f6
cc.bisect : 1.080 cc.ridder : 1.250 cc.brenth : 1.200 cc.brentq : 1.130 . ---------------------------------------------------------------------- Ran 423 tests in 129.553s OK <unittest.TextTestRunner instance at 0x85de84c> So, it seems to be a problem with the atlas3.4.1_Linux_ATHLON.tar.gz from the http://math-atlas.sourceforge.net site. Do you think this is a bug on the ATLAS binary libraries side? Thanks for the help! -- Francesc Alted PGP KeyID: 0x61C8C11F Aplications developer Public PGP key available: http://www.openlc.org/falted_at_openlc.asc Key fingerprint = 1518 38FE 3A3D 8BE8 24A0 3E5B 1328 32CC 61C8 C11F
On Mon, 14 Oct 2002, Francesc Alted wrote:
On Mon, Oct 14, 2002 at 09:08:33PM +0300, Pearu Peterson wrote:
However, scipy/system_info.py refuses to find ATLAS libraries that are incomplete. The definition of completness is as follows:
The following four files
liblapack.a, libf77blas.a, libcblas.a, libatlas.a
Yeah, this is the problem in the case of a Debian system. liblapack.a is in /usr/lib/3dnow/atlas, and the rest of the libraries are in /usr/lib/3dnow.
Yes. I am working on it to make system_info.py more Debian friendly.
So, it seems to be a problem with the atlas3.4.1_Linux_ATHLON.tar.gz from the http://math-atlas.sourceforge.net site. Do you think this is a bug on the ATLAS binary libraries side?
Actually no. They have choosed to distribute atlas libraries in their way and that's fine. Note that the Atlas project aims providing efficient replacement for the blas library, not for the lapack library. The fact that they also provide some hooks for making lapack more efficient is very useful for us but we cannot expect that they should distribute full lapack libraries. Pearu
On Mon, 14 Oct 2002, Francesc Alted wrote:
On Mon, Oct 14, 2002 at 09:08:33PM +0300, Pearu Peterson wrote:
However, scipy/system_info.py refuses to find ATLAS libraries that are incomplete. The definition of completness is as follows:
The following four files
liblapack.a, libf77blas.a, libcblas.a, libatlas.a
Yeah, this is the problem in the case of a Debian system. liblapack.a is in /usr/lib/3dnow/atlas, and the rest of the libraries are in /usr/lib/3dnow. After doing a symbolic link from /usr/lib/3dnow/liblapack.a to /usr/lib/3dnow/atlas/liblapack.a, scipy detects the libraries and everything goes smoothly:
CVS scipy should now detect Debian atlas libraries so that creating the symbolic link above would be unnecessary. Can you confirm that? Also, the rules of detecting ATLAS libraries are now more flexible: only the files libf77blas.a, libcblas.a, libatlas.a must be in the same directory. liblapack.a can be either in the same directory, or in the subdirectory atlas*, or in a completely different place, or also missing but then LAPACK sources must be available. Btw, does Redhat and other linux distributions provide atlas libraries that are optimized for certain CPUs (e.g. Debian provides sse, sse2, 3dnow optimized atlas libraries)? If yes, what is the location and the names of such libraries? While I am on it, I could add system_info support also for these cases now. Regards, Pearu
On Tue, Oct 15, 2002 at 02:16:17AM +0300, Pearu Peterson wrote:
CVS scipy should now detect Debian atlas libraries so that creating the symbolic link above would be unnecessary. Can you confirm that?
Ok. I have checked out the last version of scipy in CVS, deleted the symbolic link (I mean, /usr/lib/3dnow/liblapack.a -> /usr/lib/3dnow/atlas/liblapack.a), and here are my results: - Output of scipy_distutils/system_info.py: atlas_info: FOUND: include_dirs = ['/usr/include'] library_dirs = ['/usr/lib/3dnow/atlas', '/usr/lib/3dnow'] libraries = ['lapack', 'f77blas', 'cblas', 'atlas'] lapack_info: FOUND: libraries = ['lapack'] library_dirs = ['/usr/lib'] which seems to be good... - After compiling and installing it, I've made the next checks: $ ldd /usr/local/lib/python2.2/site-packages/scipy/linalg/clapack.so liblapack.so.2 => /usr/lib/3dnow/atlas/liblapack.so.2 (0x40030000) libf77blas.so.2 => /usr/lib/3dnow/libf77blas.so.2 (0x4055d000) libcblas.so.2 => /usr/lib/3dnow/libcblas.so.2 (0x40575000) libatlas.so.2 => /usr/lib/3dnow/libatlas.so.2 (0x40597000) libm.so.6 => /lib/libm.so.6 (0x408ff000) libc.so.6 => /lib/libc.so.6 (0x40920000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000) $ ldd /usr/local/lib/python2.2/site-packages/scipy/linalg/cblas.so liblapack.so.2 => /usr/lib/3dnow/atlas/liblapack.so.2 (0x40020000) libf77blas.so.2 => /usr/lib/3dnow/libf77blas.so.2 (0x4054d000) libcblas.so.2 => /usr/lib/3dnow/libcblas.so.2 (0x40565000) libatlas.so.2 => /usr/lib/3dnow/libatlas.so.2 (0x40587000) libm.so.6 => /lib/libm.so.6 (0x408ef000) libc.so.6 => /lib/libc.so.6 (0x40910000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000) which seems to be fine - The first import worked: Python 2.2.1 (#1, Apr 21 2002, 08:38:44) [GCC 2.95.4 20011002 (Debian prerelease)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import scipy scipy.__cvs_version__.cvs_version (1, 144, 1520, 4355)
so far so good - when doing the tests:
scipy.test(10)
it I'm getting the next (non-fatal) errors: the first: creating test suite for: scipy.common !! FAILURE building test for scipy.common <string>:1: SyntaxError: invalid syntax (test_common.py, line 77) (in ?) No test suite found for scipy.scipy_tempfile the second: **************************************************************** WARNING: Importing clapack failed with the following exception: ----------- exceptions.ImportError: /usr/local/lib/python2.2/site-packages/scipy/linalg/clapack.so: undefined symbol: clapack_sgetri ----------- See scipy/INSTALL.txt for troubleshooting. Notes: * If atlas library is not found by scipy/system_info.py, then scipy skips building clapack and uses flapack instead. **************************************************************** the third and last: creating test suite for: scipy.linalg.matfuncs !! FAILURE building test for scipy.linalg.matfuncs <string>:1: ImportError: No module named test_matfuncs (in ?) creating test suite for: scipy.linalg.basic I've run the tests four times, and these errors are always present. Normally all the tests ended OK: . ---------------------------------------------------------------------- Ran 423 tests in 118.583s OK <unittest.TextTestRunner instance at 0x866d57c> but I had this once: . ====================================================================== FAIL: check_normal (test_morestats.test_anderson) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.2/site-packages/scipy/stats/tests/test_morestats. py", line 46, in check_normal assert(scipy.all(A < crit[-2:])) AssertionError ---------------------------------------------------------------------- Ran 423 tests in 117.422s FAILED (failures=1) <unittest.TextTestRunner instance at 0x867306c> it appeared on my second try, but I was unable to reproduce it. I suppose this is not important. Sorry, I don't remember if these errors appeared on the previous version I've tested.
Also, the rules of detecting ATLAS libraries are now more flexible: only the files libf77blas.a, libcblas.a, libatlas.a must be in the same directory. liblapack.a can be either in the same directory, or in the subdirectory atlas*, or in a completely different place, or also missing but then LAPACK sources must be available.
Great!, that should facilitate the ATLAS inclusion in scipy.
Btw, does Redhat and other linux distributions provide atlas libraries that are optimized for certain CPUs (e.g. Debian provides sse, sse2, 3dnow optimized atlas libraries)? If yes, what is the location and the names of such libraries? While I am on it, I could add system_info support also for these cases now.
I personally have only experience with Debian in the last years... Cheers, -- Francesc Alted PGP KeyID: 0x61C8C11F Scientific Aplications developer Public PGP key available: http://www.openlc.org/falted_at_openlc.asc Key fingerprint = 1518 38FE 3A3D 8BE8 24A0 3E5B 1328 32CC 61C8 C11F
On Tue, 15 Oct 2002, Francesc Alted wrote:
On Tue, Oct 15, 2002 at 02:16:17AM +0300, Pearu Peterson wrote:
CVS scipy should now detect Debian atlas libraries so that creating the symbolic link above would be unnecessary. Can you confirm that?
Ok. I have checked out the last version of scipy in CVS, deleted the symbolic link (I mean, /usr/lib/3dnow/liblapack.a -> /usr/lib/3dnow/atlas/liblapack.a), and here are my results:
<snip>
lapack_info: FOUND: libraries = ['lapack'] library_dirs = ['/usr/lib']
Btw, on Debian, if atlas is present then scipy does not need that lapack or blas would be installed.
it I'm getting the next (non-fatal) errors:
the first:
creating test suite for: scipy.common !! FAILURE building test for scipy.common <string>:1: SyntaxError: invalid syntax (test_common.py, line 77)
Fixed in CVS.
the second:
**************************************************************** WARNING: Importing clapack failed with the following exception: ----------- exceptions.ImportError: /usr/local/lib/python2.2/site-packages/scipy/linalg/clapack.so: undefined symbol: clapack_sgetri ----------- See scipy/INSTALL.txt for troubleshooting. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I understand that often such comments can be ignored (can they?) but in this particular case you should follow it literally.
the third and last:
creating test suite for: scipy.linalg.matfuncs !! FAILURE building test for scipy.linalg.matfuncs <string>:1: ImportError: No module named test_matfuncs (in ?) creating test suite for: scipy.linalg.basic
It means that nobody had a change to implement the tests for scipy.linalg.matfuncs module, yet.
but I had this once:
. ====================================================================== FAIL: check_normal (test_morestats.test_anderson) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.2/site-packages/scipy/stats/tests/test_morestats. py", line 46, in check_normal assert(scipy.all(A < crit[-2:])) AssertionError
Yes, that sometimes happens. You can ignore it.
Sorry, I don't remember if these errors appeared on the previous version I've tested.
First and third error were certainly there, second error could not be there because atlas was not used, and the fourth you'll probably see also in future, occasionally. Regards, Pearu
On Tue, Oct 15, 2002 at 09:30:20PM +0300, Pearu Peterson wrote:
See scipy/INSTALL.txt for troubleshooting. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I understand that often such comments can be ignored (can they?) but in this particular case you should follow it literally.
Yeah, this was my fault. Now this is ok. Thanks for all the help! -- Francesc Alted PGP KeyID: 0x61C8C11F Scientific aplications developer Public PGP key available: http://www.openlc.org/falted_at_openlc.asc Key fingerprint = 1518 38FE 3A3D 8BE8 24A0 3E5B 1328 32CC 61C8 C11F
On Sun, 13 Oct 2002, Stephen Boulet wrote:
I've installed atlas (atlas-3.2.1 and atlas-c++-0.4.5). "/usr/lib" is in my "/etc/ld.so.conf" and I've run "ldconfig".
Did you installed ATLAS from source? If yes, then note that when giving a command make install in ATLAS source directory, it actually does not install the libraries anywhere, it just builds them. You have to copy the libraries manually where your system can find them, preferably to /usr/local/lib/atlas. Very important: before you copy, you must apply the instructions from http://math-atlas.sourceforge.net/errata.html#completelp If no, then what Linux distribution are you using? Check that the directory where ATLAS libraries were installed, contains at least the following four files: libatlas.a libcblas.a libf77blas.a liblapack.a Btw, scipy does not require atlas-c++.
When I try installing scipy (cvs), I get the messages below. Can someone tell me what I'm doing wrong?
(I did notice that there is a "/usr/bin/atlas-config" executable in the atlas distribution; I'm not sure how to use it though.)
There is no atlas-config in ATLAS-3.2.1 source distribution, it is not even created when building ATLAS. Pearu
You hit the nail on the head. I'm using Gentoo linux, which is a source based distribution. The install file used here places those files: /usr/lib/libtstatlas.a /usr/lib/libatlas.a /usr/lib/libf77blas.a /usr/lib/libcblas.a /usr/share/atlas/liblapack.a I copied /usr/share/atlas/liblapack.a to /usr/lib, reran ldconfig, and setup.py worked. Thanks! I'll submit the bug to bugs.gentoo.org. -- Stephen On Sunday 13 October 2002 12:49 pm, Pearu Peterson wrote:
On Sun, 13 Oct 2002, Stephen Boulet wrote:
I've installed atlas (atlas-3.2.1 and atlas-c++-0.4.5). "/usr/lib" is in my "/etc/ld.so.conf" and I've run "ldconfig".
Did you installed ATLAS from source?
If yes, then note that when giving a command make install in ATLAS source directory, it actually does not install the libraries anywhere, it just builds them. You have to copy the libraries manually where your system can find them, preferably to /usr/local/lib/atlas. Very important: before you copy, you must apply the instructions from http://math-atlas.sourceforge.net/errata.html#completelp
If no, then what Linux distribution are you using? Check that the directory where ATLAS libraries were installed, contains at least the following four files: libatlas.a libcblas.a libf77blas.a liblapack.a
participants (4)
-
Francesc Alted -
Pearu Peterson -
Skip Montanaro -
Stephen Boulet