Mailman 3 June 2010 - SciPy-User

corrcoef and dump
by R. Padraic Springuel 24 Jun '10

24 Jun '10

Is there a version of corrcoef in numpy that doesn't return a matrix output? I'm trying to calculate a 15x15 matrix of correlation coefficients, but my program is crashing with a MemoryError during the calculations. As a result, I'd like to break up the calculations to ease the memory load. If I feed corrcoef the lists in a pairwise fashion, then I don't need the matrix format of the output, and indeed would find it slightly easier if it wasn't matrix formated (though ultimately I can deal with the matrix format if needed). Also, if I dump an ndarray to file and that array later changes in a program, will the file be updated to reflect the changes? I didn't think so originally, but I have a program that dumps an array to file which is working in a DropBox folder and got a notice that the file was written, and then a couple of minutes later that it was updated. Since there is only one dump command in the program, this makes me suspect that the mutable nature of the array is being played with by the program (accidentally, but that's a separate issue) and then redumping the array. -- R. Padraic Springuel Research Assistant Department of Physics and Astronomy University of Maine Bennett 309 Office Hours: By Appointment Only

2 1

info scipy.ndimage.filters.maximum_filter
by David Pine 24 Jun '10

24 Jun '10

How do I get more detailed information about scipy.ndimage.filters.maximum_filter than is available at the Numpy and Scipy Documentation Reference Guide? The guide tells you what the routine does but not how it does it.

4 4

Re: [SciPy-User] Up-to-date SciPy/NumPy docs
by Pierre Raybaut 24 Jun '10

24 Jun '10

Thanks for clarifying things regarding SciPy/NumPy docs stability. >From what I understand, I think that the best for Python(x,y) is to keep distributing the outdated versions of the docs. After all, essential features are still the same and were already well documented. Thanks again for your answer. Long live to SciPy! Cheers, Pierre 2010/6/24 <scipy-user-request(a)scipy.org>: > Date: Wed, 23 Jun 2010 15:02:18 -0700 > From: David Goldsmith <d.l.goldsmith(a)gmail.com> > Subject: Re: [SciPy-User] Up-to-date SciPy/NumPy docs > To: SciPy Users List <scipy-user(a)scipy.org> > Message-ID: > <AANLkTimKqdl66Xi9MPAYdvsGIpf0CvKEP5u8li3lSQH_(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > The problem is, the doc is *not* stable: the NumPy doc is quasi-stable > (there remain 59 "documents" - docstrings, user guide pages, reference > pages, and tutorial pages - in "Being written" status, and 81 in "Needs > editing" status, 19 of which are new since the end of April, i.e., NumPy > continues to be moving target), and the SciPy doc is in a highly > "non-uniform" state, both with respect to quantity and quality, which is > putting it euphemistically IMHO: frankly, if you distribute the SciPy doc > now as it is, you're doing us a better service by telling people that it > *isn't* stable, because saying it *is* in its present state, well, that's > what would make us look unprofessional (again, IMO). More to the point, the > SciPy doc is presently the focus of an ostensibly community-wide effort to > improve it, so, again ostensibly, presently it is anything but "stable." > > There's a lot of work to be done, but, under the philosophy that some doc is > better than no doc at all, the present, dynamic state hasn't stopped us from > releasing it "as is" in the past; indeed, in the past, I believe this has > been the main source of doc improvement: a user posts a doc deficiency to > the list, and that's when it's been taken care of - the "itch-scratching" > approach. I *think* we all wish that the doc could be completed yesterday, > but the fact of the matter is, until the community steps up and decides that > writing good, clear and complete doc for the stuff that's already there is > *at least* as important as increasing the code base, the state of the doc > will forever be "unstable." > > DG > > On Wed, Jun 23, 2010 at 1:42 PM, Pierre Raybaut <pierre.raybaut(a)gmail.com>wrote: > >> Hi all, >> >> I was planning to update Python(x,y) NumPy and SciPy plugins up to >> (resp.) v1.4.1 and v0.7.2 for a long time now but I was waiting for >> the documentation to be updated as well. But when I'm going to SciPy >> documentation website, there are either still outdated versions of >> NumPy and Scipy documentation available for download. >> >> More precisely, I have the choice to download either outdated .chm >> (the most interesting format for Windows) or .pdf documentations, or >> "too recent" html versions (drafts). >> >> Wouldn't be more logical to propose a stable version of these >> documentations along with current stable releases? >> I'm tired to deliver draft versions with Python(x,y), it simply seems >> unprofessional... >> >> In other words, I would really appreciate .chm docs to be updated to >> the latest stable releases of NumPy and SciPy. I'm sure that all >> Windows scientific Python users will appreciate as well! >> >> Cheers, >> Pierre >> _______________________________________________ >> SciPy-User mailing list >> SciPy-User(a)scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-user >> > > > > -- > Mathematician: noun, someone who disavows certainty when their uncertainty > set is non-empty, even if that set has measure zero. > > Hope: noun, that delusive spirit which escaped Pandora's jar and, with her > lies, prevents mankind from committing a general suicide. (As interpreted > by Robert Graves) >

2 1

Up-to-date SciPy/NumPy docs
by Pierre Raybaut 24 Jun '10

24 Jun '10

Hi all, I was planning to update Python(x,y) NumPy and SciPy plugins up to (resp.) v1.4.1 and v0.7.2 for a long time now but I was waiting for the documentation to be updated as well. But when I'm going to SciPy documentation website, there are either still outdated versions of NumPy and Scipy documentation available for download. More precisely, I have the choice to download either outdated .chm (the most interesting format for Windows) or .pdf documentations, or "too recent" html versions (drafts). Wouldn't be more logical to propose a stable version of these documentations along with current stable releases? I'm tired to deliver draft versions with Python(x,y), it simply seems unprofessional... In other words, I would really appreciate .chm docs to be updated to the latest stable releases of NumPy and SciPy. I'm sure that all Windows scientific Python users will appreciate as well! Cheers, Pierre

5 6

Re: [SciPy-User] Up-to-date SciPy/NumPy docs
by Pierre Raybaut 24 Jun '10

24 Jun '10

Hi, > Hi, > > Wed, 23 Jun 2010 22:42:46 +0200, Pierre Raybaut wrote: > > In other words, I would really appreciate .chm docs to be updated to the > > latest stable releases of NumPy and SciPy. I'm sure that all Windows > > scientific Python users will appreciate as well! > > Note that if you wish to ship up-to-date CHM files, you should be able to > build them on Windows, in case we're too slow to respond. Of course I can. But if I had to build every single documentation for every single package included in Python(x,y)... well there won't be any Python(x,y)! I can't afford to spend so much time on each library. At some point, every library developer has to his job entirely: writing source code, building binaries and distributing them *and* doing the same for documentation. When building a module distribution like Python(x,y), my job should be to redistribute and to package things, not to build them from scratch. (Of course there are exceptions like VTK and ITK which I build every time from source.) This being said, I'm conscious that documentation may seem like a waste of time for developers, but from the user point of view documentation is almost as important as code itself. Cheers, Pierre > -- > Pauli Virtanen

1 0

OpenOpt and Scipy.optimize
by bowie_22 24 Jun '10

24 Jun '10

Hello together, during my evaluation of scipy as subsitute for Matlab I started to look at the optimization features of sciypy by looking at the optimze module. I posted a question and one answer contained a hint to OpenOpt. Now I am a little bit unsure how to proceed. Does it make more sense to look at OpenOpt rather then evaluating scipy.optimize? Regrads Marcus

2 1

calculate xcorr/acorr without plotting?
by David Ho 23 Jun '10

23 Jun '10

Hi all! Is there a way to calculate a cross-correlation (xcorr) or autocorrelation (acorr) without actually plotting a figure? For example, for histograms, I can use matplotlib.pyplot.hist() to plot a histogram, but I can also use numpy.histogram() if I just want to compute the values without plotting a figure. I'd like to do something similar for cross-correlations. Thanks for your help! --David Ho

2 1

Question about scikits.timeseries.lib.moving_funcs.mov_average
by Andreas 23 Jun '10

23 Jun '10

Hi, in the docstring to mov_average() it says: The result will also be masked at i if any of the input values in the slice ``[i-span:i+1]`` are masked Is there any way to prevent this behaviour? I have a very patchy timeseries (one value every 3 to 6 days), and I'd like to use the mov_average function to smooth these data. Any idea how to do that? mov_average(data,20) would have been perfect, if not for the masked values ... What I tried so far is mov_average(data.compressed(),20) but that has the same size as data.compressed(). I would really like to have daily values .. Thanks for your insight! Cheers, Andreas.

3 8

Lapack testing
by gintare statkute 23 Jun '10

23 Jun '10

I am trying to install Scientific Python. Have to check if LAPACK is working. According manual got several instructions, which i do not understand. from: http://www.netlib.org/lapack/lawn81/node24.html http://www.netlib.org/lapack/lawn41/index.html How to understand sentences: Compile the files xLINTSTF and either SCLNTSTF or DZLNTSTF link them to your matrix generator library, your LAPACK library, and your BLAS library For each of the test programs, associate the appropriate data file with Fortran unit number 5. Associate a suitably named file (e.g., SLINTST.OUT) with this unit number 6. Run the test programs. Under those sentences there must be certain commands: for compilation. linking, association. How to run test programs - there are tens of them in the directories /testing/link /testing/eig regards, gintare statkute

1 0

Autocorrelation function: Convolution vs FFT
by Skipper Seabold 22 Jun '10

22 Jun '10

I am trying to compute the autocorrelation via convolution and via fft and am far from an expert in DSP. I'm wondering if someone can spot anything that might introduce numerical inaccuracies or if I'm stuck with the following two being slightly different. Generate some autocorrelated data: import numpy as np nobs = 150000 x = np.zeros((nobs)) for i in range(1,nobs): x[i] = .85 * x[i-1] + np.random.randn() # compute ACF using convolution x0 = x - x.mean() # this takes a while for the big data acf1 = np.correlate(x0,x0,'full')[nobs-1:]/nobs acf1 /= acf1[0] # compute ACF using FFT Frf = np.fft.fft(x0, n=2*nobs) # zero-pad for separability Sf = Frf * Frf.conjugate() acf2 = np.fft.ifft(Sf) acf2 = acf2[1:nobs+1]/nobs acf2 /= acf2[0] acf2 = acf2.real np.linalg.norm(acf1-acf2, ord=2) They are pretty close, but I would expect them to be closer than this. np.max(acf1-acf2) 0.006581962491189159 np.min(acf1-ac2) -0.0062705596399049799 Skipper

3 5