Re: [Numpy-discussion] Biulding Numarray with atlas.
Nadav Horesh wrote:
Whenever you need a patch to compile numarray with lapack/atlas, you should check if the linear_algebra module works.
Of course. And test I have, and no, it doesn't work: ImportError: /usr/lib/python2.4/site-packages/numarray/linear_algebra/lapack_lite2.so: undefined symbol: dgesdd_ I'm guessing this is a symbol mangling mismatch between what lapack_lite2 is expecting and what my atlas install used (i.e. too many or not enough underscores) For what it's worth, I get the same error with Numeric. which brings me back to my first question: Where should I get a good lapack/blas for Fedora core 4? There are lapack and blas rpms, but there is no indication of whether they are optimized or not. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Thu, 2005-07-21 at 14:23, Chris Barker wrote:
Nadav Horesh wrote:
Whenever you need a patch to compile numarray with lapack/atlas, you should check if the linear_algebra module works.
Of course. And test I have, and no, it doesn't work:
ImportError: /usr/lib/python2.4/site-packages/numarray/linear_algebra/lapack_lite2.so: undefined symbol: dgesdd_
I'm guessing this is a symbol mangling mismatch between what lapack_lite2 is expecting and what my atlas install used (i.e. too many or not enough underscores) For what it's worth, I get the same error with Numeric.
I think this is a SciPy FAQ and following their advice is how I've worked around it in the past myself: http://www.scipy.org/documentation/mailman?fn=scipy-dev/2001-November/000105...
Todd Miller wrote:
I think this is a SciPy FAQ and following their advice is how I've worked around it in the past myself:
http://www.scipy.org/documentation/mailman?fn=scipy-dev/2001-November/000105...
I'd seem that note before, and it wasn't very helpful, but then I noticed, in the Numeric customize.py: Example: Some Linux distributions have ATLAS, but not LAPACK (if you're liblapack file is smaller than a megabyte, this is probably you) Use our own f2c'd lapack libraries, but use ATLAS for BLAS. My linux distro (FC4) has a full lapack, but it's not optimized, in fact, it's slower than lapack-lite. However, I got atlas from the atlas site, and it doesn't come with a full lapack, which would explain the missing symbols. Using: # Using ATLAS blas in /usr/local with lapack-lite if 1: use_system_lapack = 0 use_system_blas = 1 lapack_library_dirs = ['/usr/local/lib/atlas'] lapack_libraries = ['cblas', 'f77blas', 'atlas', 'g2c'] It compiles, works, and is about twice as fast as using the the Numeric-supplied blas. I still would like to find a full, optimized lapack for Fedora Core 4. On my last machine, I got about a six times speed-up using the atlas that came with Gentoo. But for the moment, I'm OK. Also, I saw something in the atlas docs about how to merge the atlas lapack with another full lapack, so maybe I can get that to work. Now on to see if I can do something similar with numarray. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
It looks like I'm managing to maintain this thread all by myself! I thought I'd post this final note, so that anyone searching the archives can see my solution. It's a bit ugly, but it seems to work. First the summary: I'm trying to build both Numeric and numarray on a new installation of Linux Fedora Core 4 on a Pentium M laptop. Fedora helpfully provides a full lapack and blas rpms, but they don't appear to be at all optimized. In fact, when I used them, it was slower than the built-in lapack-lite. I haven't been able to find a complete atlas+lapack distribution for Fedora core 4 (or anything like it), so this is what I did: Got the atlas binaries from the atlas site: https://sourceforge.net/projects/math-atlas/ I downloaded: atlas3.6.0_Linux_P4SSE2.tar.gz NOTE: these are pretty old...should I be using the 3.7.10 unstable version instead? I then merged these with the Fedora lapack (should be in /usr/lib/liblapack.a) by following the directions in the atlas README: """******** GETTING A FULL LAPACK LIB ************************** ATLAS does not provide a full lapack library. However, there is a simple way to get ATLAS to provide its faster LAPACK routines to a full LAPACK library. ATLAS's internal routines are distinct from LAPACK's, so it is safe to compile ATLAS's LAPACK routines directly into a netlib-style LAPACK library. First, obtain the LAPACK src from netlib and build the LAPACK library as normal. Then, in this directory (where you should have a liblapack.a), issue the following commands: mkdir tmp cd tmp ar x ../liblapack.a cp <your LAPACK path & lib> ../liblapack.a ar r ../liblapack.a *.o cd .. rm -rf tmp Just linking in ATLAS's liblapack.a first will not get you the best LAPACK performance, mainly because LAPACK's untuned ILAENV will be used instead of ATLAS's tuned one. """ I installed them in: /usr/local/lib/atlas and /usr/local/include/atlas Then I set up the numarray and Numeric build process to use them. For numarray, I edited cfg_packages.py, and put in: USE_LAPACK = True near the top, and edited on of the blocks to read: print "Adding paths for lapack/blas" lapack_dirs = ['/usr/local/lib/atlas'] lapack_libs = ['lapack', 'cblas', 'f77blas', 'atlas', 'g2c', 'gfortran'] lapack_include_dirs += ['/usr/local/include/atlas/'] NOTE: the gfortran lib is something I hadn't seen before, but I needed it to resolve missing symbols. I'm guess that the fedora lapack used gfortran. then a setup.py build, setup.py install, and it seems to work, and is about twice as fast as lapack-lite on my simple test case. In the past, I got about a 7 times speed-up with the atlas that I got from Gentoo. I'm a bit disappointed, but it's OK, and I'm not really doing anything where it matters much. For Numeric, I edited customize.py: # Using ATLAS blas in /usr/local if 1: use_system_lapack = 1 lapack_library_dirs = ['/usr/local/lib/atlas'] lapack_libraries = ['lapack', 'cblas', 'f77blas', 'atlas', 'g2c', 'gfortran'] And that did it. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
participants (2)
-
Chris Barker -
Todd Miller