Re: [Numpy-discussion] linalg.svd not working?
Date: Tue, 5 May 2009 09:24:53 -0600 From: Charles R Harris <charlesr.harris@gmail.com> ... This is almost always an ATLAS problem. Where did your ATLAS come from and what distro are you running?
You are probably right. I compiled and installed ATLAS from source. The distro is Redhat Enterprise Linux 4. I had to because the ones from the distro are compiled targetting 64-bit architecture. I largely followed the instructions at http://www.scipy.org/Installing_SciPy/Linux#head-eecf834fad12bf7a62575252854... . Built lapack 3.1.1, then copied the library to ATLAS and compiled per instructions. The compilers are CC='gcc -m32' version 3.4.4 (enforcing 32 bit compilation), g77 3.4.4. At various points I needed to define flags to ensure 32 bits, though not in numpy compilation. I have gfortran on the system but I didn't use it. liblapack.so and other .so files are linked to libg2c.so not libgfortran. ATLAS (3.8.3) was configured with the additional -32 flag as well. make check works nicely. What should I check in order to find the error with ATLAS configuration and/or installation? Or is there a 32 bit version binary I can download/ use (even if only for testing)? Regards, Muhammad Alkarouri
Muhammad Alkarouri wrote:
Date: Tue, 5 May 2009 09:24:53 -0600 From: Charles R Harris <charlesr.harris@gmail.com>
...
This is almost always an ATLAS problem. Where did your ATLAS come from and what distro are you running?
You are probably right. I compiled and installed ATLAS from source. The distro is Redhat Enterprise Linux 4. I had to because the ones from the distro are compiled targetting 64-bit architecture.
I largely followed the instructions at http://www.scipy.org/Installing_SciPy/Linux#head-eecf834fad12bf7a62575252854... . Built lapack 3.1.1, then copied the library to ATLAS and compiled per instructions. The compilers are CC='gcc -m32' version 3.4.4 (enforcing 32 bit compilation), g77 3.4.4. At various points I needed to define flags to ensure 32 bits, though not in numpy compilation. I have gfortran on the system but I didn't use it.
What does ldd lapack_lite.so returns (lapack_lite.so is in numpy/linalg, in your installed directory) ? It may be that numpy uses gfortran, whereas ATLAS is built with g77. gfortran and g77 should not be mixed, cheers, David
--- On Wed, 6/5/09, David Cournapeau <david@ar.media.kyoto-u.ac.jp> wrote: ...
What does ldd lapack_lite.so returns (lapack_lite.so is in numpy/linalg, in your installed directory) ? It may be that numpy uses gfortran, whereas ATLAS is built with g77. gfortran and g77 should not be mixed,
Thanks David. I went there and found that lapack_lite.so didn't link to ATLAS in the first place. So I rebuilt numpy to ensure that. Now I have: ma856388@H:linalg>ldd lapack_lite.so linux-gate.so.1 => (0xffffe000) liblapack.so => /users/d88/ma856388/lib/liblapack.so (0xf790f000) libptf77blas.so => /users/d88/ma856388/lib/libptf77blas.so (0xf78f0000) libptcblas.so => /users/d88/ma856388/lib/libptcblas.so (0xf78cf000) libatlas.so => /users/d88/ma856388/lib/libatlas.so (0xf755d000) libf77blas.so => /users/d88/ma856388/lib/libf77blas.so (0xf753e000) libcblas.so => /users/d88/ma856388/lib/libcblas.so (0xf751c000) libg2c.so.0 => /usr/lib/libg2c.so.0 (0xf74d2000) libm.so.6 => /lib/tls/libm.so.6 (0xf74af000) libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xf74a7000) libc.so.6 => /lib/tls/libc.so.6 (0xf737d000) libpthread.so.0 => /lib/tls/libpthread.so.0 (0xf736b000) /lib/ld-linux.so.2 (0x56555000) but still, test_pinv hangs using almost 100% of CPU time. Any suggestions? Regards, Muhammad Alkarouri
Muhammad Alkarouri wrote:
--- On Wed, 6/5/09, David Cournapeau <david@ar.media.kyoto-u.ac.jp> wrote: ...
What does ldd lapack_lite.so returns (lapack_lite.so is in numpy/linalg, in your installed directory) ? It may be that numpy uses gfortran, whereas ATLAS is built with g77. gfortran and g77 should not be mixed,
Thanks David. I went there and found that lapack_lite.so didn't link to ATLAS in the first place. So I rebuilt numpy to ensure that. Now I have:
Ok, so that's not a gfortran problem. As Chuck, I think that's an atlas problem (you could check by compiling without ATLAS: ATLAS=None python setup.py build after removing the build directory). Your gcc compiler is quite old, so I would not be surprised it that were related, cheers, David
--- On Wed, 6/5/09, David Cournapeau <david@ar.media.kyoto-u.ac.jp> wrote: ...
Ok, so that's not a gfortran problem. As Chuck, I think that's an atlas problem (you could check by compiling without ATLAS:
It is an atlas problem. Not that I knew how to correct it, but I was able to build numpy with a standard package blas and lapack, and the tests passed without incident. Many thanks. I guess I will leave the atlas benefits for another day. Cheers, Muhammad Alkarouri
Replying to myself, just that the experience may benefit a later user. --- On Wed, 6/5/09, Muhammad Alkarouri <malkarouri@yahoo.co.uk> wrote:
From: Muhammad Alkarouri <malkarouri@yahoo.co.uk> Subject: Re: [Numpy-discussion] linalg.svd not working? ... It is an atlas problem. Not that I knew how to correct it, but I was able to build numpy with a standard package blas and lapack, and the tests passed without incident.
After installing numpy using the standard blas/lapack on red hat enterprise 4, the tests succeeded. Scipy was then compiled and installed, but it failed its test. Specifically, some tests failed with "undefined symbolr: srotmg_". This means that the standard blas library is not complete according to past emails, so I had to go back and try installing Atlas. It turns out that for the compilers I am using (gcc, g77 3.4.4) lapack and atlas must be compiled with the option "-ffloat-store" (https://bugzilla.redhat.com/show_bug.cgi?id=138683) so after doing that, recompiling and installing numpy, numpy tests succeeded (again). The last problem was when installing scipy, I had a series of errors "NoneType object is not callable", and errors with calc_lwork. Turns out they are from a library flinalg. The solution is to compile scipy with the option: python setup.py build_ext -DUNDERSCORE_G77. Probably the reason for the last group of problems would be that scipy used gfortran in some place and that the correct action was to force the g77 compiler, but I didn't check. Anyway, the numpy and scipy tests succeeded after that. For all this installation I used CC='gcc -m32 -ffloat-store' and F77FLAGS='-m32' to get 32 bit installation on an x86_64 machine. Changing most of the other flags will mess up numpy and/or scipy installation. Many thanks, Muhammad Alkarouri
participants (2)
-
David Cournapeau -
Muhammad Alkarouri