[Numpy-discussion] numpy blas running slow: how to check that it is properly linked
David Cottrell
david.cottrell at gmail.com
Tue Sep 6 17:12:46 EDT 2011
Actually this link: http://www.scipy.org/PerformanceTips seems to
indicate that numpy.dot does use blas ...
Is there some way of running ldd on the install to see what libraries
are being pulled in?
On Tue, Sep 6, 2011 at 4:13 PM, David Cottrell <david.cottrell at gmail.com> wrote:
> Thanks, I didn't realize dot was not just calling dgemm or some
> variant which I assume would be reasonably fast. I see dgemm appears
> in the numpy code in various places such as the lapack_lite module.
>
> I ran the svd test on the solaris setup and will check the OSX run
> when back at my laptop. 8.4 seconds is slightly slower than matlab but
> still seems reasonable.
>
> $ ./test_03.py
> No ATLAS:
> (1000, 1000) (1000,) (1000, 1000)
> 8.17235898972
>
> $ cat test_03.py
> #!/usr/bin/env python3
> import numpy
> import time
>
> try:
> import numpy.core._dotblas
> print('Using ATLAS:')
> except ImportError:
> print('No ATLAS:')
>
> import numpy.linalg
>
> N = 1000
> x = numpy.random.random((N,N))
> t = time.time()
> (U, s, V) = numpy.linalg.svd(x)
> print(U.shape, s.shape, V.shape)
> # S = numpy.matrix(numpy.diag(s))
> # y = U * S * V
> #print(y.shape)
>
> print(time.time()-t)
>
>
>
>
> On Tue, Sep 6, 2011 at 2:59 PM, David Cournapeau <cournape at gmail.com> wrote:
>> On Tue, Sep 6, 2011 at 2:38 PM, David Cottrell <david.cottrell at gmail.com> wrote:
>>> I posted on stackoverflow but then noticed this message board:
>>>
>>> http://stackoverflow.com/questions/7311869/python-numpy-on-solaris-blas-slow-or-not-linked
>>>
>>> I'm reposting the full post below:
>>>
>>> Matrix-Matrix multiplies are very slow on my Solaris install (running
>>> on a sparc server) compared to my OSX install (on a laptop!). The
>>> laptop runs 100 times faster (for matrix-matrix multiplies of
>>> 3000x3000 dense random matrices of doubles).
>>>
>>> It must be because the Solaris install is not using blas, but the
>>> numpy scripts are reporting that the libs are 'found'.
>>>
>>> $python3 -c "import numpy.distutils.system_info as f; d =
>>> f.get_info('blas',0); print(d); d = f.get_info('lapack',0); print(d)"
>>> {'libraries': ['sunperf'], 'library_dirs':
>>> ['/home/$myname/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib'],
>>> 'language': 'f77'}
>>> {'libraries': ['sunmath'], 'library_dirs':
>>> ['/home/$myname/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib'],'language':
>>> 'f77'}
>>>
>>> The following import FAILS on the Solaris install but succeeds on OSX:
>>>
>>> import numpy.core._dotblas
>>>
>>> There is no ATLAS available for the Solaris install. I wouldn't think
>>> this would make such a huge different in computational efficiency.
>>
>> Actually, it will make a different for this exact case, because
>> dotblas (which implements numpy.dot) depends on ATLAS, not on BLAS
>> itself (this is a bug in numpy that I meant to fix for ages). Mac OS X
>> includes ATLAS in its accelerate framework. If BLAS/LAPACK from
>> Solaris was correctly linked, you should be able to see fast
>> operations for pretty much everything else. For example, how fast is
>> SVD or eig for a 1000x1000 matrix ?
>>
>> cheers,
>>
>> David
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>
>
>
> --
> David Cottrell
> +1 416 995 9860
> http://ca.linkedin.com/in/dcottrell
>
--
David Cottrell
+1 416 995 9860
http://ca.linkedin.com/in/dcottrell
More information about the NumPy-Discussion
mailing list