On Mon, Jul 7, 2008 at 10:31, David Cournapeau wrote:

On Mon, Jul 7, 2008 at 11:44 PM, Ondrej Certik wrote:

...

The problem is with this patch:

http://projects.scipy.org/scipy/numpy/changeset/3854

and the question that we have is:

<doko> I'd like to know, if the code was changed to only work with atlas, or if was never working. if it's the latter, then we should use atlas

_dotblas depends on cblas, not blas. IIRC, Robert said that using the cblas interface around an existing blas (for the case where ATLAS is not available) would not be effective because of change of row/column order. It may be worth checking that it is indeed not useful from a speed point of view.

I was probably talking out of my ass at the time. Fortran BLAS does have flags to handle any combination of transposition. The cblas interface just picks the right ones. We probably should relax our assumption that only ATLAS provides a cblas interface. It's not as trivial as just reverting that changeset, though. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

On Tue, Jul 8, 2008 at 11:48 AM, Tiziano Zito wrote:

Hi numpy-devs, I was the one reporting the original bug about missing ATLAS support in the debian lenny python-numpy package. AFAICT the source python-numpy package in etch (numpy version 1.0.1) does not require atlas to build _dotblas.c, only lapack is needed. If you install the resulting binary package on a system where ATLAS is present, ATLAS libraries are used instead of plain lapack. So basically it was already working before the check for ATLAS was introduced into the numpy building system. Why should ATLAS now be required?

...

It's not as trivial as just reverting that changeset, though. why is that? I mean, it was *working* before...

So just removing the two lines from numpy seems to fix the problem in Debian. So far all tests seem to run both on i386 and amd64, both with and without atlas packages installed. And it is indeed faster with the altas packages instaled, yet it doesn't need them to build. I think that's what we want, no? Ondrej

On Tue, Jul 8, 2008 at 9:15 PM, Robert Kern wrote:

On Tue, Jul 8, 2008 at 08:06, Ondrej Certik wrote:

...

On Tue, Jul 8, 2008 at 11:48 AM, Tiziano Zito wrote:

...

Hi numpy-devs, I was the one reporting the original bug about missing ATLAS support in the debian lenny python-numpy package. AFAICT the source python-numpy package in etch (numpy version 1.0.1) does not require atlas to build _dotblas.c, only lapack is needed. If you install the resulting binary package on a system where ATLAS is present, ATLAS libraries are used instead of plain lapack. So basically it was already working before the check for ATLAS was introduced into the numpy building system. Why should ATLAS now be required?

...

It's not as trivial as just reverting that changeset, though. why is that? I mean, it was *working* before...

So just removing the two lines from numpy seems to fix the problem in Debian. So far all tests seem to run both on i386 and amd64, both with and without atlas packages installed. And it is indeed faster with the altas packages instaled, yet it doesn't need them to build. I think that's what we want, no?

Can you give me more details?

Sure. :)

Was the binary built on a machine with an absent ATLAS?

Yes, the binary is always built on a machine with an absent atlas, as the package is build-conflicting with atlas.

Show me the output of ldd on _dotblas.so with both ATLAS installed and not. Can you import numpy.core._dotblas explicitly under both?

ATLAS installed: ondra@fuji:~/debian$ ldd /usr/lib/python2.5/site-packages/numpy/core/_dotblas.so linux-gate.so.1 => (0xb7fba000) libblas.so.3gf => /usr/lib/atlas/libblas.so.3gf (0xb7c19000) libgfortran.so.3 => /usr/lib/libgfortran.so.3 (0xb7b67000) libm.so.6 => /lib/i686/cmov/libm.so.6 (0xb7b40000) libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7b33000) libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb79d8000) /lib/ld-linux.so.2 (0xb7fbb000) ondra@fuji:~/debian$ python Python 2.5.2 (r252:60911, Jun 25 2008, 17:58:32) [GCC 4.3.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.

...

...

import numpy.core._dotblas

ATLAS not installed: ondra@fuji:~/debian$ ldd /usr/lib/python2.5/site-packages/numpy/core/_dotblas.so linux-gate.so.1 => (0xb7f2f000) libblas.so.3gf => /usr/lib/libblas.so.3gf (0xb7e82000) libgfortran.so.3 => /usr/lib/libgfortran.so.3 (0xb7dd0000) libm.so.6 => /lib/i686/cmov/libm.so.6 (0xb7da9000) libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7d9c000) libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb7c41000) /lib/ld-linux.so.2 (0xb7f30000) ondra@fuji:~/debian$ python Python 2.5.2 (r252:60911, Jun 25 2008, 17:58:32) [GCC 4.3.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.

...

...

import numpy.core._dotblas

Ondrej

On Tue, Jul 8, 2008 at 10:19 PM, Robert Kern wrote:

On Tue, Jul 8, 2008 at 14:47, Ondrej Certik wrote:

...

On Tue, Jul 8, 2008 at 9:15 PM, Robert Kern wrote:

...

On Tue, Jul 8, 2008 at 08:06, Ondrej Certik wrote:

...

On Tue, Jul 8, 2008 at 11:48 AM, Tiziano Zito wrote:

...

Hi numpy-devs, I was the one reporting the original bug about missing ATLAS support in the debian lenny python-numpy package. AFAICT the source python-numpy package in etch (numpy version 1.0.1) does not require atlas to build _dotblas.c, only lapack is needed. If you install the resulting binary package on a system where ATLAS is present, ATLAS libraries are used instead of plain lapack. So basically it was already working before the check for ATLAS was introduced into the numpy building system. Why should ATLAS now be required?

...

It's not as trivial as just reverting that changeset, though. why is that? I mean, it was *working* before...

So just removing the two lines from numpy seems to fix the problem in Debian. So far all tests seem to run both on i386 and amd64, both with and without atlas packages installed. And it is indeed faster with the altas packages instaled, yet it doesn't need them to build. I think that's what we want, no?

Can you give me more details?

Sure. :)

...

Was the binary built on a machine with an absent ATLAS?

Yes, the binary is always built on a machine with an absent atlas, as the package is build-conflicting with atlas.

...

Show me the output of ldd on _dotblas.so with both ATLAS installed and not. Can you import numpy.core._dotblas explicitly under both?

ATLAS installed:

ondra@fuji:~/debian$ ldd /usr/lib/python2.5/site-packages/numpy/core/_dotblas.so linux-gate.so.1 => (0xb7fba000) libblas.so.3gf => /usr/lib/atlas/libblas.so.3gf (0xb7c19000) libgfortran.so.3 => /usr/lib/libgfortran.so.3 (0xb7b67000) libm.so.6 => /lib/i686/cmov/libm.so.6 (0xb7b40000) libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7b33000) libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb79d8000) /lib/ld-linux.so.2 (0xb7fbb000) ondra@fuji:~/debian$ python Python 2.5.2 (r252:60911, Jun 25 2008, 17:58:32) [GCC 4.3.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.

...

...

...

import numpy.core._dotblas

ATLAS not installed:

ondra@fuji:~/debian$ ldd /usr/lib/python2.5/site-packages/numpy/core/_dotblas.so linux-gate.so.1 => (0xb7f2f000) libblas.so.3gf => /usr/lib/libblas.so.3gf (0xb7e82000) libgfortran.so.3 => /usr/lib/libgfortran.so.3 (0xb7dd0000) libm.so.6 => /lib/i686/cmov/libm.so.6 (0xb7da9000) libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7d9c000) libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb7c41000) /lib/ld-linux.so.2 (0xb7f30000) ondra@fuji:~/debian$ python Python 2.5.2 (r252:60911, Jun 25 2008, 17:58:32) [GCC 4.3.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.

...

...

...

import numpy.core._dotblas

Okay, it turns out that libblas on Ubuntu (and I'm guessing Debian) includes the CBLAS interface.

$ nm /usr/lib/libblas.a | grep "T cblas_" 00000000 T cblas_caxpy 00000000 T cblas_ccopy ...

This is specific to Debian and its derivatives. Not all libblas's have this. So I stand by my statement that just reverting the change is not acceptable. We need a real check for the CBLAS interface. In the

Right.

meantime, the Debian package maintainer can patch the file to remove that check during the build for Debian systems.

Yes, I just did that. Thanks for the clarification. Ondrej