On Fri, 15 Oct 2004, Nick Arnett wrote:
What sort of performance should I expect using lingalg.svd on matrixes on the order of 500 x 15000? I'm using a 2 Ghz P4 system with 2 GB of memory and these are taking hours to complete. I previously was using an external SVD utility that was taking a couple of minutes, and I'm going to have to switch back to it unless there's some way to get Scipy to do this a lot faster. I'm using the pre-built binaries, with ATLAS and F2PY installed. I'm not eager to build my own, but if that is likely to speed things up a lot, I'll give it a shot.
from scipy import * from scipy_test.testing import jiffies t=jiffies();s=linalg.svd(rand(500,15000),compute_uv=0); print (jiffies()-t)/100,'sec' 22 sec t=jiffies();u,s,vh=linalg.svd(rand(500,15000),compute_uv=1); print (jiffies()-t)/100,'sec' 345 sec import scipy.linalg.atlas_version ATLAS version 3.7.8 built by pearu on Thu Sep 30 23:29:06 EEST 2004: UNAME : Linux opt 2.6.3-gentoo-r2 #1 SMP Sat Aug 28 12:40:32 EEST 2004 x86_64 AMD Opteron(tm) Processor 244 AuthenticAMD GNU/Linux INSTFLG : MMDEF : /usr/local/src/ATLAS-3.7.8/CONFIG/ARCHS/HAMMER64SSE2/gcc/gemm ARCHDEF : /usr/local/src/ATLAS-3.7.8/CONFIG/ARCHS/HAMMER64SSE2/gcc/misc F2CDEFS : -DAdd_ -DStringSunStyle CACHEEDGE: 917504 F77 : /usr/x86_64-pc-linux-gnu/gcc-bin/3.3/g77, version GNU Fortran (GCC) 3.3.4 20040623 (Gentoo Linux 3.3.4-r1, ssp-3.3.2-2,
On a system with AMD Athlon(tm) 64 Processor 3000+ and 2GB memory: - calculating full svd of a 500x15000 matrix takes 5m55s consuming about 1.8g memory - calculating only singular values takes 22s consuming about 0.2g memory Note that the above results do not contain time spent for swaping when memory is low. E.g. on a 1GB system calculating full svd would take forever.. HTH, Pearu Python 2.3.3 (#1, Aug 28 2004, 06:56:51) [GCC 3.3.3 20040412 (Gentoo Linux 3.3.3-r6, ssp-3.3.2-2, pie-8.7.6)] on linux2 Type "help", "copyright", "credits" or "license" for more information. pie-8.7.6) F77FLAGS : -fomit-frame-pointer -O -m64 -fno-second-underscore -fPIC CC : /usr/bin/gcc, version gcc (GCC) 3.3.4 20040623 (Gentoo Linux 3.3.4-r1, ssp-3.3.2-2, pie-8.7.6) CC FLAGS : -fomit-frame-pointer -O -mfpmath=387 -m64 -fPIC MCC : /usr/bin/gcc, version gcc (GCC) 3.3.4 20040623 (Gentoo Linux 3.3.4-r1, ssp-3.3.2-2, pie-8.7.6) MCCFLAGS : -fomit-frame-pointer -O -mfpmath=387 -m64 -fPIC