Hi, Today I was reading "Special Session: Making Python attractive to General Scientists (Harrington, Greenfield)" notes in http://www.scipy.org/wikis/scipy04/ConferenceSchedule and I was a bit surprised on one of the conclusions that maybe ATLAS optimization should be undone due to difficulties in building ATLAS. Though, IMHO, building recent versions of ATLAS libraries is not difficult at all on Linux platforms and not even on MS Windows (there are step-by-step instructions available on Scipy site), it just may be a very time consuming process ;). It's not even an issue for Mac as Scipy uses its vecLib framework. I can't say much on the situation on other unix platforms such as irix, sun, etc due to the lack of access to such platforms. But most of current and potential Scipy users are either on MS Windows, Linux, or Mac anyway.. But that was not the point I was surprised on. It was acctually the fact that people seem to be unaware of the possibility to build Scipy without ATLAS dependency by using Fortran sources of BLAS and LAPACK libraries. Let me stress that nothing in Scipy requires specifically ATLAS libraries, the corresponding interface in scipy.linalg is smart enough to pick up ATLAS optimized routines when available and use Fortran BLAS/LAPACK routines when they are not. My point is that there is (almost) nothing to do to "undo ATLAS optimization" in Scipy. ATLAS is optional already. However, when ATLAS is not available then Scipy needs BLAS/LAPACK libraries that currently must be provided by the system or users must download them from Netlib. I think that BLAS/LAPACK libraries are the only external libraries that Scipy currently depends on. To get rid of this dependency, I'd suggest include the sources of BLAS/LAPACK libraries to Scipy, and use them silently when optimized BLAS/LAPACK libraries are not available. This would be very similar to scipy.fftpack that silently uses Fortran fftpack sources when FFTW libraries are not found. Just wanted to clear some things up.. Pearu
Pearu Peterson wrote: [snip]
It's not even an issue for Mac as Scipy uses its vecLib framework.
vecLib, which *is* ATLAS under the covers, unfortunately does not contain the row-major versions of functions. I still always link against a full ATLAS library, but vecLib makes a nice backup for those that don't want to spend a day compiling ATLAS. -- Robert Kern rkern@ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter
Pearu Peterson wrote:
Hi,
Today I was reading "Special Session: Making Python attractive to General Scientists (Harrington, Greenfield)" notes in
http://www.scipy.org/wikis/scipy04/ConferenceSchedule
and I was a bit surprised on one of the conclusions that maybe ATLAS optimization should be undone due to difficulties in building ATLAS.
Though, IMHO, building recent versions of ATLAS libraries is not difficult at all on Linux platforms and not even on MS Windows (there are step-by-step instructions available on Scipy site), it just may be a very time consuming process ;). It's not even an issue for Mac as Scipy uses its vecLib framework. I can't say much on the situation on other unix platforms such as irix, sun, etc due to the lack of access to such platforms. But most of current and potential Scipy users are either on MS Windows, Linux, or Mac anyway..
But that was not the point I was surprised on. It was acctually the fact that people seem to be unaware of the possibility to build Scipy without ATLAS dependency by using Fortran sources of BLAS and LAPACK libraries. Let me stress that nothing in Scipy requires specifically ATLAS libraries, the corresponding interface in scipy.linalg is smart enough to pick up ATLAS optimized routines when available and use Fortran BLAS/LAPACK routines when they are not.
My point is that there is (almost) nothing to do to "undo ATLAS optimization" in Scipy. ATLAS is optional already. However, when ATLAS is not available then Scipy needs BLAS/LAPACK libraries that currently must be provided by the system or users must download them from Netlib. I think that BLAS/LAPACK libraries are the only external libraries that Scipy currently depends on.
To get rid of this dependency, I'd suggest include the sources of BLAS/LAPACK libraries to Scipy, and use them silently when optimized BLAS/LAPACK libraries are not available. This would be very similar to scipy.fftpack that silently uses Fortran fftpack sources when FFTW libraries are not found.
Just wanted to clear some things up.. Pearu
As one of the notetakers I'll admit I don't have much personal experience with the issue so I could have gotten the conversation wrong. But I think the gist was that some felt that there should be easy-to-install binary packaging for all popular platforms, not easy-to-build packaging (though of course one hopes that is also available). If having an optimized ATLAS got in the way of that goal, then it was argued that it would be better to have a slow version of the linear algebra libraries in the easy-to-install binaries. Those that needed the optimized versions could build it themselves. The stress was on making it very painless for people to install the general setup on their machines and that meant binary installers. If the conclusion that optimized atlas made that harder (either on the people doing the binary packaging--thus it never happens-- or the people doing the installing) is wrong, then I should correct the notes. But I think that was the point of the comments. Perry
participants (3)
-
Pearu Peterson -
Perry Greenfield -
Robert Kern