Mailman 3 February 2007 - SciPy-User

FFT speed ?
by Samuel GARCIA Feb. 8, 2007

Feb. 8, 2007

Hello list, I have a problem with fft speed. could someone try this : from scipy import * import time a = rand((186888));t1=time.time();fft(a);t2=time.time();t2-t1 # for me about 0.67791390419006348 s. on centrino 1.6GHz a = rand((186890));t1=time.time();fft(a);t2=time.time();t2-t1 # for me about 1.622053861618042 s. on centrino 1.6GHz a = rand((186889));t1=time.time();fft(a);t2=time.time();t2-t1 # the computation is infinit !!!! (no time to wait) Question : why is there a problem specialy for that size (186889) of fft ? It is not the only size with that problem. Thank Sam -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Samuel Garcia Universite Claude Bernard LYON 1 CNRS - UMR5020, Laboratoire des Neurosciences et Systemes Sensoriels 50, avenue Tony Garnier 69366 LYON Cedex 07 FRANCE Tél : 04 37 28 74 64 Fax : 04 37 28 76 01 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

8 16

Re: [SciPy-user] SciPy-user Digest, Vol 42, Issue 10
by Paul Ray Feb. 7, 2007

Feb. 7, 2007

On Feb 7, 2007, at 9:04 AM, scipy-user-request(a)scipy.org wrote: > I have a problem with fft speed. The speed of most FFT algorithms depends greatly on the largest prime factor of N. Here are the factors of your example numbers: >factor 186888 186888: 2 2 2 3 13 599 >factor 186890 186890: 2 5 11 1699 >factor 186889 186889: 186889 Note that 186889 is prime! This is the worst case situation! Numbers which are powers of 2 can be FFT'ed in order N*Log_2(N) time, while prime numbers take order N**2 time. This is what you are seeing. You might do a survey of FFT algorithms to see which ones perform best on prime Ns. Many you will find, work ONLY on power of 2 Ns! You should make sure that your SciPy is using FFTW3 for the FFT engine for sure, since I think it does about as well as possible on prime Ns. You can read about FFTW at http://fftw.org I rarely use FFT in SciPy/NumPy since there are many confusing statements in the instructions about FFTW2/FFTW3 support which don't make sense. The API changed from FFTW2 to FFTW3 and it is really important to get this right. I think FFTW2 support should be removed and someone should do a check to make sure the FFTW3 support is implemented correctly. Cheers, -- Paul

3 2

errors with scipy.stats
by Joshua Petterson Feb. 7, 2007

Feb. 7, 2007

5 4

Re: [SciPy-user] build issue on 64-bit Intel Core2 Duo
by Nadav Horesh Feb. 7, 2007

Feb. 7, 2007

> Few more informations: > >python numpy/distutils/cpuinfo.py: > >CPU information: getNCPUs=4 has_mmx has_sse has_sse2 is_64bit is_Intel >is_XEON is_Xeon is_i686 Well, Crossing our systems' info it looks to me that the best and straight forward way to identify Intel's imitation to amd64 arch is: def _is_Nocona(self): return self.is_64bit() and self.is_Intel() and self.is_i686() Did not test it but it fits and looks logical. Nadav.

5 4

Re: [SciPy-user] back with a problem in scipy installation...
by Eric Emsellem Feb. 7, 2007

Feb. 7, 2007

Hi again (sorry for the second post: the first one was scrubbed), I now went through a full install of atlas, lapack, blas, numpy, scipy again. I still have some failure in scipy (none in numpy except for the umfpack which I haven't installed) you can find below. I also include the explicit site.cfg that I had to twiggle in order for scipy to find atlas (no Environment variable or other way did the work for some reason): it looks weird if I compare it to "advised" site.cfg but this is the only option that recognised atlas properly. any help is welcome as always. thanks again Eric ======================= below is a summary of: 1/ the failure in scipy 2/ my site.cfg used for building numpy and scipy *************************************************************************************************** ******** Output of scipy.test() = (starting with some failure due to the lack of umfpack and then..): .................... Warning: FAILURE importing tests for <module 'scipy.linsolve.umfpack.umfpack' from '...y/linsolve/umfpack/umfpack.pyc'> /usr/local/lib/python2.4/site-packages/scipy/linsolve/umfpack/tests/test_umfpack.py:17: AttributeError: 'module' object has no attribute 'umfpack' (in ?) ................................ Found 4 tests for scipy.io.recaster Warning: FAILURE importing tests for <module 'scipy.linsolve.umfpack' from '.../linsolve/umfpack/__init__.pyc'> /usr/local/lib/python2.4/site-packages/scipy/linsolve/umfpack/tests/test_umfpack.py:17: AttributeError: 'module' object has no attribute 'umfpack' (in ?) ................... ..zswap:n=4 ..zswap:n=3 ..................Residual: 1.05006950608e-07 ......................................................................................... ====================================================================== FAIL: check_syevr (scipy.lib.tests.test_lapack.test_flapack_float) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.4/site-packages/scipy/lib/lapack/tests/esv_tests.py", line 41, in check_syevr assert_array_almost_equal(w,exact_w) File "/usr/local/lib/python2.4/site-packages/numpy/testing/utils.py", line 230, in assert_array_almost_equal header='Arrays are not almost equal') File "/usr/local/lib/python2.4/site-packages/numpy/testing/utils.py", line 215, in assert_array_compare assert cond, msg AssertionError: Arrays are not almost equal (mismatch 33.3333333333%) x: array([-0.66992444, 0.48769444, 9.18222618], dtype=float32) y: array([-0.66992434, 0.48769389, 9.18223045]) ====================================================================== FAIL: check_syevr_irange (scipy.lib.tests.test_lapack.test_flapack_float) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.4/site-packages/scipy/lib/lapack/tests/esv_tests.py", line 66, in check_syevr_irange assert_array_almost_equal(w,exact_w[rslice]) File "/usr/local/lib/python2.4/site-packages/numpy/testing/utils.py", line 230, in assert_array_almost_equal header='Arrays are not almost equal') File "/usr/local/lib/python2.4/site-packages/numpy/testing/utils.py", line 215, in assert_array_compare assert cond, msg AssertionError: Arrays are not almost equal (mismatch 33.3333333333%) x: array([-0.66992444, 0.48769444, 9.18222618], dtype=float32) y: array([-0.66992434, 0.48769389, 9.18223045]) ---------------------------------------------------------------------- Ran 1619 tests in 5.764s FAILED (failures=2) *************************************************************************************************** ******** My site.cfg used for both numpy and scipy ******** it finds atlas, but that's all. If I uncomment the lines on lapack_opt and blas_opt, it finds the [DEFAULT] library_dirs = /usr/local/lib/atlas include_dirs = /usr/local/lib/atlas [atlas] library_dirs = /usr/local/lib/atlas atlas_libs = lapack, f77blas, cblas, atlas # [blas_opt] # library_dirs = /usr/local/lib/atlas # blas_libs = fblas, f77blas, cblas # [lapack_opt] # library_dirs = /usr/local/lib/atlas # lapack_libs = lapack [blas] library_dirs = /usr/local/lib/atlas libraries = f77blas, cblas, atlas # [lapack] library_dirs = /usr/local/lib/atlas libraries = lapack, f77blas, cblas, atlas [fftw] libraries = fftw3 -- ==================================================================== Eric Emsellem emsellem(a)obs.univ-lyon1.fr Centre de Recherche Astrophysique de Lyon 9 av. Charles-Andre tel: +33 (0)4 78 86 83 84 69561 Saint-Genis Laval Cedex fax: +33 (0)4 78 86 83 86 France http://www-obs.univ-lyon1.fr/eric.emsellem ====================================================================

1 0

cblas_dtrsm not found
by Eric Emsellem Feb. 7, 2007

Feb. 7, 2007

Hi! I have been using numpy+scipy+matplotlib extensively in scripts since quite some time, and regularly updated all these packages using svn. For some reason the following does not work anymore, and is trying to find cblas_dtrsm. I have it in /usr/local/lib/atlas/libcblas.a (I checked doing a "ar -t" of that library). I also have that directory in my ld.so.conf (and indeed as ATLAS default dir). it must be a very stupid mistake I am making here, but I cannot make it work. (it always worked, and I haven't changed anything in my ATLAS/BLAS, etc) (I tried recompiling scipy but no luck) Anyone has an idea of what is wrong with that? thanks for any help here. Eric ==================================== ---> 21 from scipy import optimize, special, interpolate 22 from scipy.optimize import leastsq, fsolve 23 from scipy.special import gammainc /usr/local/lib/python2.4/site-packages/scipy/optimize/__init__.py 9 from zeros import * 10 from anneal import * ---> 11 from lbfgsb import fmin_l_bfgs_b 12 from tnc import fmin_tnc 13 from cobyla import fmin_cobyla /usr/local/lib/python2.4/site-packages/scipy/optimize/lbfgsb.py 28 29 from numpy import zeros, float64, array, int32 ---> 30 import _lbfgsb 31 import optimize 32 ImportError: /usr/local/lib/python2.4/site-packages/scipy/optimize/_lbfgsb.so: undefined symbol: cblas_dtrsm

3 14

find(M) equivalent for sparse matrices?
by David Warde-Farley Feb. 6, 2007

Feb. 6, 2007

Hi there, I'm porting some Matlab code to Numpy and having some trouble. Basically, in the matlab code we have a line that does a find() on a sparse matrix and grabs out the rows, columns and values of the non- zero element positions. It then modifies each element in a manner dependent on its row and column, and then stores all the elements back into their original spots. I can't figure out a way to do this, as comparisons don't seem to be implemented for the sparse matrix classes. I suppose I should mention I'm using Python 2.4, scipy 0.5.1, numpy 1.0. Anyone got an idea of how to do this? By the way, what I'm doing can be expressed as left and right multiplying it by a (the same on both sides) diagonal matrix, but this appears to take a lot longer than it should when I write it that way (with the diagonal stored as a sparse). David

1 0

error importing scipy
by Chiara Caronna Feb. 5, 2007

Feb. 5, 2007

Hello, since a couple of days I got an error when I try to run "from scipy import *" here is the error message.... " Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/local/lib/python2.4/site-packages/scipy/linalg/__init__.py", line 8, in ? from basic import * File "/usr/local/lib/python2.4/site-packages/scipy/linalg/basic.py", line 227, in ? import decomp File "/usr/local/lib/python2.4/site-packages/scipy/linalg/decomp.py", line 21, in ? from blas import get_blas_funcs File "/usr/local/lib/python2.4/site-packages/scipy/linalg/blas.py", line 14, in ? from scipy.linalg import fblas ImportError: /usr/local/lib/python2.4/site-packages/scipy/linalg/fblas.so: undefined symbol: srotmg_" does anyone have a clue?! thanks, Chiara _________________________________________________________________ Express yourself instantly with MSN Messenger! Download today it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

7 17

Getting started wiki page
by Gael Varoquaux Feb. 5, 2007

Feb. 5, 2007

Once again I had to explain to a new student what scipy was, what python was, and what he should read in order to start working with these tools. I have started a "Getting Started" page on the wiki to answer this problem. The idea would be to guide the complete beginner to get him productive as quickly as possible (like the "Getting started" page in matlab). Currently the page is quite empty and pretty bad, but if people start improving it it could be very useful and hopefuly be linked from the front page. The page is not linked by any page yet, but it is accessible at: http://scipy.org/Getting_Started Cheers, Gaël

5 6

RHEL 4 SciPy Install Failure
by Chad Kidder Feb. 4, 2007

Feb. 4, 2007

I was trying to install SciPy today after a successful numpy install and it went well until it got to the fortran compilation of dfftpack. Below is an exerpt of the error. Anyone know what's going on? running build_clib customize UnixCCompiler customize UnixCCompiler using build_clib customize GnuFCompiler customize GnuFCompiler customize GnuFCompiler using build_clib building 'dfftpack' library compiling Fortran sources Fortran f77 compiler: /usr/bin/g77 -g -Wall -fno-second-underscore -fPIC -O3 -funroll-loops -march=i686 -mmmx -msse2 -msse -fomit-frame-pointer creating build/temp.linux-x86_64-2.3 creating build/temp.linux-x86_64-2.3/Lib creating build/temp.linux-x86_64-2.3/Lib/fftpack creating build/temp.linux-x86_64-2.3/Lib/fftpack/dfftpack compile options: '-c' g77:f77: Lib/fftpack/dfftpack/dsinqi.f Lib/fftpack/dfftpack/dsinqi.f:0: error: CPU you selected does not support x86-64 instruction set Lib/fftpack/dfftpack/dsinqi.f:0: error: CPU you selected does not support x86-64 instruction set Lib/fftpack/dfftpack/dsinqi.f:0: error: CPU you selected does not support x86-64 instruction set Lib/fftpack/dfftpack/dsinqi.f:0: error: CPU you selected does not support x86-64 instruction set error: Command "/usr/bin/g77 -g -Wall -fno-second-underscore -fPIC -O3 -funroll-loops -march=i686 -mmmx -msse2 -msse -fomit-frame-pointer -c -c Lib/fftpack/dfftpack/dsinqi.f -o build/temp.linux-x86_64-2.3/Lib/fftpack/dfftpack/dsinqi.o" failed with exit status 1 Thanks for your help.

2 1