numpy blas running slow: how to check that it is properly linked
I posted on stackoverflow but then noticed this message board: http://stackoverflow.com/questions/7311869/python-numpy-on-solaris-blas-slow... I'm reposting the full post below: Matrix-Matrix multiplies are very slow on my Solaris install (running on a sparc server) compared to my OSX install (on a laptop!). The laptop runs 100 times faster (for matrix-matrix multiplies of 3000x3000 dense random matrices of doubles). It must be because the Solaris install is not using blas, but the numpy scripts are reporting that the libs are 'found'. $python3 -c "import numpy.distutils.system_info as f; d = f.get_info('blas',0); print(d); d = f.get_info('lapack',0); print(d)" {'libraries': ['sunperf'], 'library_dirs': ['/home/$myname/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib'], 'language': 'f77'} {'libraries': ['sunmath'], 'library_dirs': ['/home/$myname/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib'],'language': 'f77'} The following import FAILS on the Solaris install but succeeds on OSX: import numpy.core._dotblas There is no ATLAS available for the Solaris install. I wouldn't think this would make such a huge different in computational efficiency. Any suggestions for other tests? Does the distutils.system_info not necessarily mean anything about the install? I tried the numpy.alterdot() command but that seems to have no effect. Even more notes: And I basically followed the setup on http://www.scipy.org/Installing_SciPy/Solaris, except that my site.cfg looked like: [DEFAULT] library_dirs = $PATH_TO_MY_SOLARIS_DIR/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib [blas] blas_libs = sunperf [lapack] lapack_libs = sunmath
On Tue, Sep 6, 2011 at 2:38 PM, David Cottrell <david.cottrell@gmail.com> wrote:
I posted on stackoverflow but then noticed this message board:
http://stackoverflow.com/questions/7311869/python-numpy-on-solaris-blas-slow...
I'm reposting the full post below:
Matrix-Matrix multiplies are very slow on my Solaris install (running on a sparc server) compared to my OSX install (on a laptop!). The laptop runs 100 times faster (for matrix-matrix multiplies of 3000x3000 dense random matrices of doubles).
It must be because the Solaris install is not using blas, but the numpy scripts are reporting that the libs are 'found'.
$python3 -c "import numpy.distutils.system_info as f; d = f.get_info('blas',0); print(d); d = f.get_info('lapack',0); print(d)" {'libraries': ['sunperf'], 'library_dirs': ['/home/$myname/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib'], 'language': 'f77'} {'libraries': ['sunmath'], 'library_dirs': ['/home/$myname/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib'],'language': 'f77'}
The following import FAILS on the Solaris install but succeeds on OSX:
import numpy.core._dotblas
There is no ATLAS available for the Solaris install. I wouldn't think this would make such a huge different in computational efficiency.
Actually, it will make a different for this exact case, because dotblas (which implements numpy.dot) depends on ATLAS, not on BLAS itself (this is a bug in numpy that I meant to fix for ages). Mac OS X includes ATLAS in its accelerate framework. If BLAS/LAPACK from Solaris was correctly linked, you should be able to see fast operations for pretty much everything else. For example, how fast is SVD or eig for a 1000x1000 matrix ? cheers, David
Thanks, I didn't realize dot was not just calling dgemm or some variant which I assume would be reasonably fast. I see dgemm appears in the numpy code in various places such as the lapack_lite module. I ran the svd test on the solaris setup and will check the OSX run when back at my laptop. 8.4 seconds is slightly slower than matlab but still seems reasonable. $ ./test_03.py No ATLAS: (1000, 1000) (1000,) (1000, 1000) 8.17235898972 $ cat test_03.py #!/usr/bin/env python3 import numpy import time try: import numpy.core._dotblas print('Using ATLAS:') except ImportError: print('No ATLAS:') import numpy.linalg N = 1000 x = numpy.random.random((N,N)) t = time.time() (U, s, V) = numpy.linalg.svd(x) print(U.shape, s.shape, V.shape) # S = numpy.matrix(numpy.diag(s)) # y = U * S * V #print(y.shape) print(time.time()-t) On Tue, Sep 6, 2011 at 2:59 PM, David Cournapeau <cournape@gmail.com> wrote:
On Tue, Sep 6, 2011 at 2:38 PM, David Cottrell <david.cottrell@gmail.com> wrote:
I posted on stackoverflow but then noticed this message board:
http://stackoverflow.com/questions/7311869/python-numpy-on-solaris-blas-slow...
I'm reposting the full post below:
Matrix-Matrix multiplies are very slow on my Solaris install (running on a sparc server) compared to my OSX install (on a laptop!). The laptop runs 100 times faster (for matrix-matrix multiplies of 3000x3000 dense random matrices of doubles).
It must be because the Solaris install is not using blas, but the numpy scripts are reporting that the libs are 'found'.
$python3 -c "import numpy.distutils.system_info as f; d = f.get_info('blas',0); print(d); d = f.get_info('lapack',0); print(d)" {'libraries': ['sunperf'], 'library_dirs': ['/home/$myname/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib'], 'language': 'f77'} {'libraries': ['sunmath'], 'library_dirs': ['/home/$myname/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib'],'language': 'f77'}
The following import FAILS on the Solaris install but succeeds on OSX:
import numpy.core._dotblas
There is no ATLAS available for the Solaris install. I wouldn't think this would make such a huge different in computational efficiency.
Actually, it will make a different for this exact case, because dotblas (which implements numpy.dot) depends on ATLAS, not on BLAS itself (this is a bug in numpy that I meant to fix for ages). Mac OS X includes ATLAS in its accelerate framework. If BLAS/LAPACK from Solaris was correctly linked, you should be able to see fast operations for pretty much everything else. For example, how fast is SVD or eig for a 1000x1000 matrix ?
cheers,
David _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
-- David Cottrell +1 416 995 9860 http://ca.linkedin.com/in/dcottrell
Actually this link: http://www.scipy.org/PerformanceTips seems to indicate that numpy.dot does use blas ... Is there some way of running ldd on the install to see what libraries are being pulled in? On Tue, Sep 6, 2011 at 4:13 PM, David Cottrell <david.cottrell@gmail.com> wrote:
Thanks, I didn't realize dot was not just calling dgemm or some variant which I assume would be reasonably fast. I see dgemm appears in the numpy code in various places such as the lapack_lite module.
I ran the svd test on the solaris setup and will check the OSX run when back at my laptop. 8.4 seconds is slightly slower than matlab but still seems reasonable.
$ ./test_03.py No ATLAS: (1000, 1000) (1000,) (1000, 1000) 8.17235898972
$ cat test_03.py #!/usr/bin/env python3 import numpy import time
try: import numpy.core._dotblas print('Using ATLAS:') except ImportError: print('No ATLAS:')
import numpy.linalg
N = 1000 x = numpy.random.random((N,N)) t = time.time() (U, s, V) = numpy.linalg.svd(x) print(U.shape, s.shape, V.shape) # S = numpy.matrix(numpy.diag(s)) # y = U * S * V #print(y.shape)
print(time.time()-t)
On Tue, Sep 6, 2011 at 2:59 PM, David Cournapeau <cournape@gmail.com> wrote:
On Tue, Sep 6, 2011 at 2:38 PM, David Cottrell <david.cottrell@gmail.com> wrote:
I posted on stackoverflow but then noticed this message board:
http://stackoverflow.com/questions/7311869/python-numpy-on-solaris-blas-slow...
I'm reposting the full post below:
Matrix-Matrix multiplies are very slow on my Solaris install (running on a sparc server) compared to my OSX install (on a laptop!). The laptop runs 100 times faster (for matrix-matrix multiplies of 3000x3000 dense random matrices of doubles).
It must be because the Solaris install is not using blas, but the numpy scripts are reporting that the libs are 'found'.
$python3 -c "import numpy.distutils.system_info as f; d = f.get_info('blas',0); print(d); d = f.get_info('lapack',0); print(d)" {'libraries': ['sunperf'], 'library_dirs': ['/home/$myname/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib'], 'language': 'f77'} {'libraries': ['sunmath'], 'library_dirs': ['/home/$myname/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib'],'language': 'f77'}
The following import FAILS on the Solaris install but succeeds on OSX:
import numpy.core._dotblas
There is no ATLAS available for the Solaris install. I wouldn't think this would make such a huge different in computational efficiency.
Actually, it will make a different for this exact case, because dotblas (which implements numpy.dot) depends on ATLAS, not on BLAS itself (this is a bug in numpy that I meant to fix for ages). Mac OS X includes ATLAS in its accelerate framework. If BLAS/LAPACK from Solaris was correctly linked, you should be able to see fast operations for pretty much everything else. For example, how fast is SVD or eig for a 1000x1000 matrix ?
cheers,
David _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
-- David Cottrell +1 416 995 9860 http://ca.linkedin.com/in/dcottrell
-- David Cottrell +1 416 995 9860 http://ca.linkedin.com/in/dcottrell
On Tue, Sep 6, 2011 at 5:12 PM, David Cottrell <david.cottrell@gmail.com> wrote:
Actually this link: http://www.scipy.org/PerformanceTips seems to indicate that numpy.dot does use blas ...
This not true (you can check by looking into numpy/core/setup.py, which explicitly checks for ATLAS for _dotblas). The idea is that numpy can be built without a fortran compiler, but linking against blas requires a fortran compiler. By depending on ATLAS, we can still get a fast numpy.dot in the cases where a fortran compiler is not available. But strictly speaking, the code itself only depends on CBLAS availability. The issue being that some CBLAS libraries still require a fortran compiler to be linked against...
Is there some way of running ldd on the install to see what libraries are being pulled in?
Yes, just use ldd on the .so inside a numpy installation. Generally, python extensions are simply binaries that be be dlopened, cheers, David
On 06.09.2011, at 22:13, David Cottrell wrote:
Thanks, I didn't realize dot was not just calling dgemm or some variant which I assume would be reasonably fast. I see dgemm appears in the numpy code in various places such as the lapack_lite module.
I ran the svd test on the solaris setup and will check the OSX run when back at my laptop. 8.4 seconds is slightly slower than matlab but still seems reasonable.
$ ./test_03.py No ATLAS: (1000, 1000) (1000,) (1000, 1000) 8.17235898972
I just ran your benchmark code on OSX 10.7.1 on an 2011 MacBook Pro (core-i7) with numpy.version.version '2.0.0.dev-900d82e': Using ATLAS: ((1000, 1000), (1000,), (1000, 1000)) 0.908223152161 cheers, Samuel
Thanks, just getting back to this. I just checked again, and after setting my LD_LIBRARY_PATH properly, ldd shows _dotblas.so pointing and the sunmath and sunperf libraries. However the test_03.py still runs at about 8-9 seconds ... far too slow. ~/local/lib/python3.1/site-packages/numpy/core $ ldd _dotblas.so | sed -e 's/$me/$USERNAME/g' libsunperf.so.8 => /home/$USERNAME/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib//libsunperf.so.8 libsunmath.so.1 => /home/$USERNAME/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib//libsunmath.so.1 libgcc_s.so.1 => /usr/sfw/lib/libgcc_s.so.1 libfsu.so.1 => /home/$USERNAME/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib//libfsu.so.1 libfui.so.2 => /home/$USERNAME/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib//libfui.so.2 libpicl.so.1 => /usr/lib/libpicl.so.1 libmtsk.so.1 => /lib/libmtsk.so.1 libm.so.2 => /lib/libm.so.2 libc.so.1 => /lib/libc.so.1 libm.so.1 => /lib/libm.so.1 libdl.so.1 => /lib/libdl.so.1 libdoor.so.1 => /lib/libdoor.so.1 libthread.so.1 => /lib/libthread.so.1 libkstat.so.1 => /lib/libkstat.so.1 libpthread.so.1 => /lib/libpthread.so.1 librt.so.1 => /lib/librt.so.1 libaio.so.1 => /lib/libaio.so.1 libmd.so.1 => /lib/libmd.so.1 /platform/SUNW,Sun-Fire-V490/lib/libc_psr.so.1 /platform/SUNW,Sun-Fire-V490/lib/libmd_psr.so.1 ~/local/lib/python3.1/site-packages/numpy/core $ ~/python/numpy/B/test_03.py No ATLAS: 8.49377894402 (1000, 1000) (1000,) (1000, 1000) On Wed, Sep 7, 2011 at 9:08 AM, Samuel John <scipy@samueljohn.de> wrote:
On 06.09.2011, at 22:13, David Cottrell wrote:
Thanks, I didn't realize dot was not just calling dgemm or some variant which I assume would be reasonably fast. I see dgemm appears in the numpy code in various places such as the lapack_lite module.
I ran the svd test on the solaris setup and will check the OSX run when back at my laptop. 8.4 seconds is slightly slower than matlab but still seems reasonable.
$ ./test_03.py No ATLAS: (1000, 1000) (1000,) (1000, 1000) 8.17235898972
I just ran your benchmark code on OSX 10.7.1 on an 2011 MacBook Pro (core-i7) with numpy.version.version '2.0.0.dev-900d82e': Using ATLAS: ((1000, 1000), (1000,), (1000, 1000)) 0.908223152161
cheers, Samuel _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
-- David Cottrell +1 416 995 9860 http://ca.linkedin.com/in/dcottrell
The blas implementation you are using may be slow. Here's my ldd on _dotblas.so, that shows it is using libblas (this is on Ubuntu 11.04): linux-vdso.so.1 => (0x00007fffad5ff000) libblas.so.3gf => /usr/lib/libblas.so.3gf (0x00007fc608ea4000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so. 6 (0x00007fc608b10000) libgfortran.so.3 => /usr/lib/x86_64-linux-gnu/libgfortran.so.3 (0x00007fc60882b000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fc6085a6000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fc608390000) /lib64/ld-linux-x86-64.so.2 (0x00007fc609352000) My guess is BLAS is implemented in libsunmath.so but is a slow implementation. It's really just a guess though... -=- Olivier 2011/9/20 David Cottrell <david.cottrell@gmail.com>
Thanks, just getting back to this. I just checked again, and after setting my LD_LIBRARY_PATH properly, ldd shows _dotblas.so pointing and the sunmath and sunperf libraries. However the test_03.py still runs at about 8-9 seconds ... far too slow.
~/local/lib/python3.1/site-packages/numpy/core $ ldd _dotblas.so | sed -e 's/$me/$USERNAME/g' libsunperf.so.8 =>
/home/$USERNAME/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib//libsunperf.so.8 libsunmath.so.1 =>
/home/$USERNAME/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib//libsunmath.so.1 libgcc_s.so.1 => /usr/sfw/lib/libgcc_s.so.1 libfsu.so.1 =>
/home/$USERNAME/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib//libfsu.so.1 libfui.so.2 =>
/home/$USERNAME/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib//libfui.so.2 libpicl.so.1 => /usr/lib/libpicl.so.1 libmtsk.so.1 => /lib/libmtsk.so.1 libm.so.2 => /lib/libm.so.2 libc.so.1 => /lib/libc.so.1 libm.so.1 => /lib/libm.so.1 libdl.so.1 => /lib/libdl.so.1 libdoor.so.1 => /lib/libdoor.so.1 libthread.so.1 => /lib/libthread.so.1 libkstat.so.1 => /lib/libkstat.so.1 libpthread.so.1 => /lib/libpthread.so.1 librt.so.1 => /lib/librt.so.1 libaio.so.1 => /lib/libaio.so.1 libmd.so.1 => /lib/libmd.so.1 /platform/SUNW,Sun-Fire-V490/lib/libc_psr.so.1 /platform/SUNW,Sun-Fire-V490/lib/libmd_psr.so.1
~/local/lib/python3.1/site-packages/numpy/core $ ~/python/numpy/B/test_03.py No ATLAS: 8.49377894402 (1000, 1000) (1000,) (1000, 1000)
On Wed, Sep 7, 2011 at 9:08 AM, Samuel John <scipy@samueljohn.de> wrote:
On 06.09.2011, at 22:13, David Cottrell wrote:
Thanks, I didn't realize dot was not just calling dgemm or some variant which I assume would be reasonably fast. I see dgemm appears in the numpy code in various places such as the lapack_lite module.
I ran the svd test on the solaris setup and will check the OSX run when back at my laptop. 8.4 seconds is slightly slower than matlab but still seems reasonable.
$ ./test_03.py No ATLAS: (1000, 1000) (1000,) (1000, 1000) 8.17235898972
I just ran your benchmark code on OSX 10.7.1 on an 2011 MacBook Pro
(core-i7) with numpy.version.version '2.0.0.dev-900d82e':
Using ATLAS: ((1000, 1000), (1000,), (1000, 1000)) 0.908223152161
cheers, Samuel _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
-- David Cottrell +1 416 995 9860 http://ca.linkedin.com/in/dcottrell _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Tue, Sep 20, 2011 at 9:56 AM, David Cottrell <david.cottrell@gmail.com> wrote:
Thanks, just getting back to this. I just checked again, and after setting my LD_LIBRARY_PATH properly, ldd shows _dotblas.so pointing and the sunmath and sunperf libraries. However the test_03.py still runs at about 8-9 seconds ... far too slow.
~/local/lib/python3.1/site-packages/numpy/core $ ldd _dotblas.so | sed -e 's/$me/$USERNAME/g' libsunperf.so.8 => /home/$USERNAME/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib//libsunperf.so.8 libsunmath.so.1 => /home/$USERNAME/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib//libsunmath.so.1 libgcc_s.so.1 => /usr/sfw/lib/libgcc_s.so.1 libfsu.so.1 => /home/$USERNAME/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib//libfsu.so.1 libfui.so.2 => /home/$USERNAME/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib//libfui.so.2 libpicl.so.1 => /usr/lib/libpicl.so.1 libmtsk.so.1 => /lib/libmtsk.so.1 libm.so.2 => /lib/libm.so.2 libc.so.1 => /lib/libc.so.1 libm.so.1 => /lib/libm.so.1 libdl.so.1 => /lib/libdl.so.1 libdoor.so.1 => /lib/libdoor.so.1 libthread.so.1 => /lib/libthread.so.1 libkstat.so.1 => /lib/libkstat.so.1 libpthread.so.1 => /lib/libpthread.so.1 librt.so.1 => /lib/librt.so.1 libaio.so.1 => /lib/libaio.so.1 libmd.so.1 => /lib/libmd.so.1 /platform/SUNW,Sun-Fire-V490/lib/libc_psr.so.1 /platform/SUNW,Sun-Fire-V490/lib/libmd_psr.so.1
~/local/lib/python3.1/site-packages/numpy/core $ ~/python/numpy/B/test_03.py No ATLAS: 8.49377894402 (1000, 1000) (1000,) (1000, 1000)
As mentioned earlier, we currently only optimize dot if you are using ATLAS, not any generic BLAS implementation. Your linear algebra functions will be fast, though, cheers, David
The test_03.py was basically a linalg.svd test (which I think is a linalg only routine?"). I guess for linalg checking, I should run ldd on lapack_lite.so? (included below). It's sounding like I need to get ATLAS up and running, but I'm still puzzled as to why the svd routine seems to be so much slower on the solaris install. As a reminder, test_03.py was something like this. It's also somewhere earlier in the thread. import numpy.linalg N = 1000 x = numpy.random.random((N,N)) t = time.time() (U, s, V) = numpy.linalg.svd(x) # S = numpy.matrix(numpy.diag(s)) # y = U * S * V #print(y.shape) print(time.time()-t) print(U.shape, s.shape, V.shape) $ ldd lapack_lite.so libsunmath.so.1 => /opt/SUNWspro11/SUNWspro/lib/libsunmath.so.1 libsunperf.so.6 => /opt/SUNWspro11/SUNWspro/lib/v8plus/libsunperf.so.6 libfsu.so.1 => /opt/SUNWspro11/SUNWspro/lib/v8plus/libfsu.so.1 libm.so.1 => /lib/libm.so.1 libc.so.1 => /lib/libc.so.1 libfsu.so.1 => /opt/SUNWspro11/SUNWspro/prod/lib/v8/../libfsu.so.1 libfui.so.2 => /opt/SUNWspro11/SUNWspro/prod/lib/v8/../libfui.so.2 libmtsk.so.1 => /lib/libmtsk.so.1 libthread.so.1 => /lib/libthread.so.1 libkstat.so.1 => /lib/libkstat.so.1 libdl.so.1 => /lib/libdl.so.1 libpthread.so.1 => /lib/libpthread.so.1 libm.so.2 => /lib/libm.so.2 librt.so.1 => /lib/librt.so.1 libaio.so.1 => /lib/libaio.so.1 libmd.so.1 => /lib/libmd.so.1 /opt/SUNWspro11/SUNWspro/prod/lib/v8/../cpu/sparcv9+vis2/libsunperf_isa.so.6 /opt/SUNWspro11/SUNWspro/prod/lib/v8/../cpu/sparcv8plus+vis/libsunperf_isa.so.6 /opt/SUNWspro11/SUNWspro/prod/lib/v8plus/../cpu/sparcv9+vis2/libfsu_isa.so.1 /platform/SUNW,Sun-Fire-V490/lib/libc_psr.so.1 /platform/SUNW,Sun-Fire-V490/lib/libmd_psr.so.1 On Tue, Sep 20, 2011 at 1:18 PM, David Cournapeau <cournape@gmail.com> wrote:
On Tue, Sep 20, 2011 at 9:56 AM, David Cottrell <david.cottrell@gmail.com> wrote:
Thanks, just getting back to this. I just checked again, and after setting my LD_LIBRARY_PATH properly, ldd shows _dotblas.so pointing and the sunmath and sunperf libraries. However the test_03.py still runs at about 8-9 seconds ... far too slow.
~/local/lib/python3.1/site-packages/numpy/core $ ldd _dotblas.so | sed -e 's/$me/$USERNAME/g' libsunperf.so.8 => /home/$USERNAME/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib//libsunperf.so.8 libsunmath.so.1 => /home/$USERNAME/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib//libsunmath.so.1 libgcc_s.so.1 => /usr/sfw/lib/libgcc_s.so.1 libfsu.so.1 => /home/$USERNAME/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib//libfsu.so.1 libfui.so.2 => /home/$USERNAME/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib//libfui.so.2 libpicl.so.1 => /usr/lib/libpicl.so.1 libmtsk.so.1 => /lib/libmtsk.so.1 libm.so.2 => /lib/libm.so.2 libc.so.1 => /lib/libc.so.1 libm.so.1 => /lib/libm.so.1 libdl.so.1 => /lib/libdl.so.1 libdoor.so.1 => /lib/libdoor.so.1 libthread.so.1 => /lib/libthread.so.1 libkstat.so.1 => /lib/libkstat.so.1 libpthread.so.1 => /lib/libpthread.so.1 librt.so.1 => /lib/librt.so.1 libaio.so.1 => /lib/libaio.so.1 libmd.so.1 => /lib/libmd.so.1 /platform/SUNW,Sun-Fire-V490/lib/libc_psr.so.1 /platform/SUNW,Sun-Fire-V490/lib/libmd_psr.so.1
~/local/lib/python3.1/site-packages/numpy/core $ ~/python/numpy/B/test_03.py No ATLAS: 8.49377894402 (1000, 1000) (1000,) (1000, 1000)
As mentioned earlier, we currently only optimize dot if you are using ATLAS, not any generic BLAS implementation. Your linear algebra functions will be fast, though,
cheers,
David _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
-- David Cottrell +1 416 995 9860 http://ca.linkedin.com/in/dcottrell
On Tue, Sep 20, 2011 at 3:18 PM, David Cottrell <david.cottrell@gmail.com> wrote:
The test_03.py was basically a linalg.svd test (which I think is a linalg only routine?"). I guess for linalg checking, I should run ldd on lapack_lite.so? (included below).
It's sounding like I need to get ATLAS up and running, but I'm still puzzled as to why the svd routine seems to be so much slower on the solaris install.
What's the CPU of your solaris machine and macbook ? For macbook, the info as given by the system profiler are enough, and maybe psrinfo -pv is enough on solaris. cheers, David
On Tue, Sep 20, 2011 at 1:18 PM, David Cournapeau <cournape@gmail.com> wrote:
On Tue, Sep 20, 2011 at 9:56 AM, David Cottrell <david.cottrell@gmail.com> wrote:
Thanks, just getting back to this. I just checked again, and after setting my LD_LIBRARY_PATH properly, ldd shows _dotblas.so pointing and the sunmath and sunperf libraries. However the test_03.py still runs at about 8-9 seconds ... far too slow.
~/local/lib/python3.1/site-packages/numpy/core $ ldd _dotblas.so | sed -e 's/$me/$USERNAME/g' libsunperf.so.8 => /home/$USERNAME/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib//libsunperf.so.8 libsunmath.so.1 => /home/$USERNAME/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib//libsunmath.so.1 libgcc_s.so.1 => /usr/sfw/lib/libgcc_s.so.1 libfsu.so.1 => /home/$USERNAME/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib//libfsu.so.1 libfui.so.2 => /home/$USERNAME/local/archive/SolarisStudio12.2-solaris-sparc-tar-ML/solstudio12.2/lib//libfui.so.2 libpicl.so.1 => /usr/lib/libpicl.so.1 libmtsk.so.1 => /lib/libmtsk.so.1 libm.so.2 => /lib/libm.so.2 libc.so.1 => /lib/libc.so.1 libm.so.1 => /lib/libm.so.1 libdl.so.1 => /lib/libdl.so.1 libdoor.so.1 => /lib/libdoor.so.1 libthread.so.1 => /lib/libthread.so.1 libkstat.so.1 => /lib/libkstat.so.1 libpthread.so.1 => /lib/libpthread.so.1 librt.so.1 => /lib/librt.so.1 libaio.so.1 => /lib/libaio.so.1 libmd.so.1 => /lib/libmd.so.1 /platform/SUNW,Sun-Fire-V490/lib/libc_psr.so.1 /platform/SUNW,Sun-Fire-V490/lib/libmd_psr.so.1
If this refers to the model of your solaris machine (SunFire V490, meaning UltraSparc IV), then the numbers are actually not hard to believe. Those are almost a decade-old CPU (from 2003-2004), and SPARC are not speed deamons in the first place. Having them 10 times slower than a recent core i7 (which are very performant for floating points) seems "reasonable". cheers, David
participants (4)
-
David Cottrell -
David Cournapeau -
Olivier Delalleau -
Samuel John