Mailman 3 May 2014 - NumPy-Discussion

incremental histogram
by Neal Becker 07 May '14

07 May '14

2 1

Numpy 1.9.x has been branched
by Charles R Harris 07 May '14

07 May '14

Hi All, Numpy 1.9.x is now official. Until the first beta is released we will only be committing bug fixes to master. The first beta should be out in 5-7 days if all goes well. Meanwhile, it would be useful if as many people as possible tested the branch to discover all the obvious problems that somehow escaped our notice. Think of it as an alpha release. TIA, Chuck

1 0

repeat an array without allocation
by srean 05 May '14

05 May '14

Hi all, is there an efficient way to do the following without allocating A where A = np.repeat(x, [4, 2, 1, 3], axis=0) c = A.dot(b) # b.shape thanks -- srean

3 4

ANN: Scipy 0.14.0 release
by Ralf Gommers 04 May '14

04 May '14

Hi, On behalf of the Scipy development team I'm pleased to announce the availability of Scipy 0.14.0. This release contains new features (see release notes below) and 8 months worth of maintenance work. 80 people contributed to this release. This is also the first release for which binary wheels are available on PyPi for OS X, supporting the python.org Python. Wheels for Windows are still being worked on, those may follow at a later date. This release requires Python 2.6, 2.7 or 3.2-3.4 and NumPy 1.5.1 or greater. Sources and binaries can be found at https://sourceforge.net/projects/scipy/files/scipy/0.14.0/. Enjoy, Ralf ========================== SciPy 0.14.0 Release Notes ========================== .. contents:: SciPy 0.14.0 is the culmination of 8 months of hard work. It contains many new features, numerous bug-fixes, improved test coverage and better documentation. There have been a number of deprecations and API changes in this release, which are documented below. All users are encouraged to upgrade to this release, as there are a large number of bug-fixes and optimizations. Moreover, our development attention will now shift to bug-fix releases on the 0.14.x branch, and on adding new features on the master branch. This release requires Python 2.6, 2.7 or 3.2-3.4 and NumPy 1.5.1 or greater. New features ============ ``scipy.interpolate`` improvements ---------------------------------- A new wrapper function `scipy.interpolate.interpn` for interpolation on regular grids has been added. `interpn` supports linear and nearest-neighbor interpolation in arbitrary dimensions and spline interpolation in two dimensions. Faster implementations of piecewise polynomials in power and Bernstein polynomial bases have been added as `scipy.interpolate.PPoly` and `scipy.interpolate.BPoly`. New users should use these in favor of `scipy.interpolate.PiecewisePolynomial`. `scipy.interpolate.interp1d` now accepts non-monotonic inputs and sorts them. If performance is critical, sorting can be turned off by using the new ``assume_sorted`` keyword. Functionality for evaluation of bivariate spline derivatives in ``scipy.interpolate`` has been added. The new class `scipy.interpolate.Akima1DInterpolator` implements the piecewise cubic polynomial interpolation scheme devised by H. Akima. Functionality for fast interpolation on regular, unevenly spaced grids in arbitrary dimensions has been added as `scipy.interpolate.RegularGridInterpolator` . ``scipy.linalg`` improvements ----------------------------- The new function `scipy.linalg.dft` computes the matrix of the discrete Fourier transform. A condition number estimation function for matrix exponential, `scipy.linalg.expm_cond`, has been added. ``scipy.optimize`` improvements ------------------------------- A set of benchmarks for optimize, which can be run with ``optimize.bench()``, has been added. `scipy.optimize.curve_fit` now has more controllable error estimation via the ``absolute_sigma`` keyword. Support for passing custom minimization methods to ``optimize.minimize()`` and ``optimize.minimize_scalar()`` has been added, currently useful especially for combining ``optimize.basinhopping()`` with custom local optimizer routines. ``scipy.stats`` improvements ---------------------------- A new class `scipy.stats.multivariate_normal` with functionality for multivariate normal random variables has been added. A lot of work on the ``scipy.stats`` distribution framework has been done. Moment calculations (skew and kurtosis mainly) are fixed and verified, all examples are now runnable, and many small accuracy and performance improvements for individual distributions were merged. The new function `scipy.stats.anderson_ksamp` computes the k-sample Anderson-Darling test for the null hypothesis that k samples come from the same parent population. ``scipy.signal`` improvements ----------------------------- ``scipy.signal.iirfilter`` and related functions to design Butterworth, Chebyshev, elliptical and Bessel IIR filters now all use pole-zero ("zpk") format internally instead of using transformations to numerator/denominator format. The accuracy of the produced filters, especially high-order ones, is improved significantly as a result. The new function `scipy.signal.vectorstrength` computes the vector strength, a measure of phase synchrony, of a set of events. ``scipy.special`` improvements ------------------------------ The functions `scipy.special.boxcox` and `scipy.special.boxcox1p`, which compute the Box-Cox transformation, have been added. ``scipy.sparse`` improvements ----------------------------- - Significant performance improvement in CSR, CSC, and DOK indexing speed. - When using Numpy >= 1.9 (to be released in MM 2014), sparse matrices function correctly when given to arguments of ``np.dot``, ``np.multiply`` and other ufuncs. With earlier Numpy and Scipy versions, the results of such operations are undefined and usually unexpected. - Sparse matrices are no longer limited to ``2^31`` nonzero elements. They automatically switch to using 64-bit index data type for matrices containing more elements. User code written assuming the sparse matrices use int32 as the index data type will continue to work, except for such large matrices. Code dealing with larger matrices needs to accept either int32 or int64 indices. Deprecated features =================== ``anneal`` ---------- The global minimization function `scipy.optimize.anneal` is deprecated. All users should use the `scipy.optimize.basinhopping` function instead. ``scipy.stats`` --------------- ``randwcdf`` and ``randwppf`` functions are deprecated. All users should use distribution-specific ``rvs`` methods instead. Probability calculation aliases ``zprob``, ``fprob`` and ``ksprob`` are deprecated. Use instead the ``sf`` methods of the corresponding distributions or the ``special`` functions directly. ``scipy.interpolate`` --------------------- ``PiecewisePolynomial`` class is deprecated. Backwards incompatible changes ============================== scipy.special.lpmn ------------------ ``lpmn`` no longer accepts complex-valued arguments. A new function ``clpmn`` with uniform complex analytic behavior has been added, and it should be used instead. scipy.sparse.linalg ------------------- Eigenvectors in the case of generalized eigenvalue problem are normalized to unit vectors in 2-norm, rather than following the LAPACK normalization convention. The deprecated UMFPACK wrapper in ``scipy.sparse.linalg`` has been removed due to license and install issues. If available, ``scikits.umfpack`` is still used transparently in the ``spsolve`` and ``factorized`` functions. Otherwise, SuperLU is used instead in these functions. scipy.stats ----------- The deprecated functions ``glm``, ``oneway`` and ``cmedian`` have been removed from ``scipy.stats``. ``stats.scoreatpercentile`` now returns an array instead of a list of percentiles. scipy.interpolate ----------------- The API for computing derivatives of a monotone piecewise interpolation has changed: if `p` is a ``PchipInterpolator`` object, `p.derivative(der)` returns a callable object representing the derivative of `p`. For in-place derivatives use the second argument of the `__call__` method: `p(0.1, der=2)` evaluates the second derivative of `p` at `x=0.1`. The method `p.derivatives` has been removed. Other changes ============= Authors ======= * Marc Abramowitz + * Anders Bech Borchersen + * Vincent Arel-Bundock + * Petr Baudis + * Max Bolingbroke * François Boulogne * Matthew Brett * Lars Buitinck * Evgeni Burovski * CJ Carey + * Thomas A Caswell + * Pawel Chojnacki + * Phillip Cloud + * Stefano Costa + * David Cournapeau * David Menendez Hurtado + * Matthieu Dartiailh + * Christoph Deil + * Jörg Dietrich + * endolith * Francisco de la Peña + * Ben FrantzDale + * Jim Garrison + * André Gaul * Christoph Gohlke * Ralf Gommers * Robert David Grant * Alex Griffing * Blake Griffith * Yaroslav Halchenko * Andreas Hilboll * Kat Huang * Gert-Ludwig Ingold * James T. Webber + * Dorota Jarecka + * Todd Jennings + * Thouis (Ray) Jones * Juan Luis Cano Rodríguez * ktritz + * Jacques Kvam + * Eric Larson + * Justin Lavoie + * Denis Laxalde * Jussi Leinonen + * lemonlaug + * Tim Leslie * Alain Leufroy + * George Lewis + * Max Linke + * Brandon Liu + * Benny Malengier + * Matthias Kümmerer + * Cimarron Mittelsteadt + * Eric Moore * Andrew Nelson + * Niklas Hambüchen + * Joel Nothman + * Clemens Novak * Emanuele Olivetti + * Stefan Otte + * peb + * Josef Perktold * pjwerneck * poolio * Jérôme Roy + * Carl Sandrock + * Andrew Sczesnak + * Shauna + * Fabrice Silva * Daniel B. Smith * Patrick Snape + * Thomas Spura + * Jacob Stevenson * Julian Taylor * Tomas Tomecek * Richard Tsai * Jacob Vanderplas * Joris Vankerschaver + * Pauli Virtanen * Warren Weckesser A total of 80 people contributed to this release. People with a "+" by their names contributed a patch for the first time. This list of names is automatically generated, and may not be fully complete.

1 0

Re: [Numpy-discussion] Second order gradient in numpy
by RayS 03 May '14

03 May '14

I recently tried diff and gradient for some medical time domain data, and the result nearly looked like pure noise. I just found this after seeing John Agosta's post https://gist.github.com/mblondel/487187 """ Find the solution for the second order differential equation u'' = -u with u(0) = 10 and u'(0) = -5 using the Euler and the Runge-Kutta methods. This works by splitting the problem into 2 first order differential equations u' = v v' = f(t,u) with u(0) = 10 and v(0) = -5 """ - Ray At 12:19 PM 5/2/2014, you wrote: >On Thu, May 1, 2014 at 6:00 PM, Yuxiang Wang ><<mailto:yw5aj@virginia.edu>yw5aj(a)virginia.edu> wrote: >Thank you for your input! I prefer np.gradient because it takes >mid-point finite difference estimation instead of one-sided estimates, >but np.diff() is also a good idea. Just wondering why np.gradient does >not have something similar, being curious :) > > >well, according to the docs, the second order >diff() is just calling diff twice anyway, so >really the same as what you've done with >gradient anyway. I suspect it's just that no one >bothered to add that to the gradient API. > >BTW, I think that numy can handle piecewise >polynomials (i.e. splines), so depending on the >noisiness of your data, a cubic spline fit may >give better gradients -- and if your data are >noise second order gradients can get *really* noisy > >-CHB > > > >Â >Shawn > >On Thu, May 1, 2014 at 6:42 PM, Christian K. ><<mailto:ckkart@hoc.net>ckkart(a)hoc.net> wrote: > > Am 01.05.14 18:45, schrieb Yuxiang Wang: > >> Hi all, > >> > >> I am trying to calculate the 2nd-order gradient numerically of an > >> array in numpy. > >> > >> Â Â Â import numpy as np > >> Â Â Â a = np.sin(np.arange(0, 10, .01)) > >> Â Â Â da = np.gradient(a) > >> Â Â Â dda = np.gradient(da) > > > > It looks like you are looking for the derivative rather than the > > gradient. Have a look at: > > > > np.diff(a, n=1, axis=-1) > > > > n is the order if the derivative. > > > > Christian > > > > > > _______________________________________________ > > NumPy-Discussion mailing list > > <mailto:NumPy-Discussion@scipy.org>NumPy-Discussion(a)scipy.org > > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > > >-- >Yuxiang "Shawn" Wang >Gerling Research Lab >University of Virginia ><mailto:yw5aj@virginia.edu>yw5aj(a)virginia.edu ><tel:%2B1%20%28434%29%20284-0836>+1 (434) 284-0836 >https://sites.google.com/a/virginia.edu/yw5aj/ >_______________________________________________ >NumPy-Discussion mailing list ><mailto:NumPy-Discussion@scipy.org>NumPy-Discussion(a)scipy.org >http://mail.scipy.org/mailman/listinfo/numpy-discussion > > > > >-- > >Christopher Barker, Ph.D. >Oceanographer > >Emergency Response Division >NOAA/NOS/OR&R Â Â Â Â Â Â (206) 526-6959Â Â voice >7600 Sand Point Way NE Â Â (206) 526-6329Â Â fax >Seattle, WA Â 98115 Â Â Â Â (206) 526-6317Â Â main reception > ><mailto:Chris.Barker@noaa.gov>Chris.Barker(a)noaa.gov >_______________________________________________ >NumPy-Discussion mailing list >NumPy-Discussion(a)scipy.org >http://mail.scipy.org/mailman/listinfo/numpy-discussion

1 0

Second order gradient in numpy
by Yuxiang Wang 02 May '14

02 May '14

Hi all, I am trying to calculate the 2nd-order gradient numerically of an array in numpy. import numpy as np a = np.sin(np.arange(0, 10, .01)) da = np.gradient(a) dda = np.gradient(da) This is what I come up. Is the the way it should be done? I am asking this, because in numpy there isn't an option saying np.gradient(a, order=2). I am concerned about whether this usage is wrong, and that is why numpy does not have this implemented. Thank you! -Shawn -- Yuxiang "Shawn" Wang Gerling Research Lab University of Virginia yw5aj(a)virginia.edu +1 (434) 284-0836 https://sites.google.com/a/virginia.edu/yw5aj/

4 6

ANN: HDF5 for Python 2.3.0
by Andrew Collette 02 May '14

02 May '14

Announcing HDF5 for Python (h5py) 2.3.0 ======================================= The h5py team is happy to announce the availability of h5py 2.3.0 (final). Thanks to everyone who provided beta feedback! What's h5py? ------------ The h5py package is a Pythonic interface to the HDF5 binary data format. It lets you store huge amounts of numerical data, and easily manipulate that data from NumPy. For example, you can slice into multi-terabyte datasets stored on disk, as if they were real NumPy arrays. Thousands of datasets can be stored in a single file, categorized and tagged however you want. Changes ------- This release introduces some important new features, including: * Support for arbitrary vlen data * Improved exception messages * Improved setuptools support * Multiple additions to the low-level API * Improved support for MPI features * Single-step build for HDF5 on Windows Major fixes since beta: * LZF compression crash on Win64 * Unhelpful error message relating to chunked storage * Import error for IPython completer on certain platforms A complete description of changes is available online: http://docs.h5py.org/en/latest/whatsnew/2.3.html Where to get it --------------- Downloads, documentation, and more are available at the h5py website: http://www.h5py.org Acknowledgements ---------------- The h5py package relies on third-party testing and contributions. For the 2.3 release, thanks especially to: * Martin Teichmann * Florian Rathgerber * Pierre de Buyl * Thomas Caswell * Andy Salnikov * Darren Dale * Robert David Grant * Toon Verstraelen * Many others who contributed bug reports and testing

3 4

Re: [Numpy-discussion] arrays and : behaviour
by did did 01 May '14

01 May '14

thanks, it works well !!!! see you and thanks again ----- Original Message ----- From: Sebastian Berg Sent: 05/01/14 03:54 PM To: numpy-discussion(a)scipy.org Subject: Re: [Numpy-discussion] arrays and : behaviour On Do, 2014-05-01 at 09:45 -0400, Benjamin Root wrote: > By default, the hold is already True. In fact, that might explain some > of the differences in what you are seeing. There are more points in > the second image than in the first one, so I wonder if you are seeing > some leftovers of previous plot commands? > > > One issue I do see is that the slicing is incorrect. [0:3] means index > 0, 1, and 2. So index 3 is never accessed. I think you want [0:4]. > > > I should also note that once you have your data as a numpy array, your > indexing can be greatly simplified: > plt.plot(array4D[0][0][0:4][0],array4D[0][0][0:4][0],'bo') > > can be done as: > plt.plot(array4D[0, 0, 0:4, 0], array4D[0, 0, 0:4, 0], 'bo') > Yeah, also arr[0:4][0] is the same as arr[0]. So you actually *must* use an array and the arr[0:4, 0] way if you want to do indexing like that... - Sebastian > > Cheers! > Ben Root > > > > On Thu, May 1, 2014 at 8:33 AM, Eelco Hoogendoorn > <hoogendoorn.eelco(a)gmail.com> wrote: > You problem isn't with colon indexing, but with the > interpretation of the arguments to plot. multiple calls to > plot with scalar arguments do not have the same result as a > single call with array arguments. For this to work as > intended, you would need plt.hold(True), for starters, and > maybe there are other subtleties. > > > On Thu, May 1, 2014 at 1:31 PM, did did <21did21(a)gmx.com> > wrote: > > Hello all and sorry for my bad english, > > i am a beginner with python and i try to save a lot of > data in several folders in a 4D matrix > and then to plot two columns of this 4D matrix. > > Bellow, i have the code to fill my 4D matrix, it works > very well : > > [CODE]matrix4D=[] > for i in Numbers: > readInFolder=folderPrefixe+i+"/" > matrix3D=[] > for j in listeOfdata: > nameOfFile=filePrefixe+i+"-"+j+extensionTXT > nameOfFile=readInFolder+nameOfFile > > matrix2D=np.loadtxt(nameOfFile,delimiter=",",skiprows=1) > matrix3D.append(matrix2D) > matrix4D.append(matrix3D) > array4D = np.asarray(matrix4D)[/CODE] > > But now, i want to plot the third column as function > of the third too (just for trying) and i use > this stupid manner that works well : > > [CODE]plt.figure(1) > temp=plt.plot(array4D[0][0][0][0],array4D[0][0][0][0],'bo') > temp=plt.plot(array4D[0][0][1][0],array4D[0][0][1][0],'bo') > temp=plt.plot(array4D[0][0][2][0],array4D[0][0][2][0],'bo') > temp=plt.plot(array4D[0][0][3][0],array4D[0][0][3][0],'bo') > plt.show()[/CODE] > > Now, i want to use a more smart manner and i use ":" > like this > > [CODE]plt.figure(1) > temp=plt.plot(array4D[0][0][0:3][0],array4D[0][0][0:3][0],'bo') > plt.show()[/CODE] > > The result should be the same but i don't got the same > results!!! > > In attachement you have the two corresponding plots, > can you explain to me with > i don't have the same plots ?? > > thanks for all > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion

1 0

Re: [Numpy-discussion] arrays and : behaviour
by did did 01 May '14

01 May '14

Thanks all for your help! i will try bye ;-) ----- Original Message ----- From: Sebastian Berg Sent: 05/01/14 03:54 PM To: numpy-discussion(a)scipy.org Subject: Re: [Numpy-discussion] arrays and : behaviour On Do, 2014-05-01 at 09:45 -0400, Benjamin Root wrote: > By default, the hold is already True. In fact, that might explain some > of the differences in what you are seeing. There are more points in > the second image than in the first one, so I wonder if you are seeing > some leftovers of previous plot commands? > > > One issue I do see is that the slicing is incorrect. [0:3] means index > 0, 1, and 2. So index 3 is never accessed. I think you want [0:4]. > > > I should also note that once you have your data as a numpy array, your > indexing can be greatly simplified: > plt.plot(array4D[0][0][0:4][0],array4D[0][0][0:4][0],'bo') > > can be done as: > plt.plot(array4D[0, 0, 0:4, 0], array4D[0, 0, 0:4, 0], 'bo') > Yeah, also arr[0:4][0] is the same as arr[0]. So you actually *must* use an array and the arr[0:4, 0] way if you want to do indexing like that... - Sebastian > > Cheers! > Ben Root > > > > On Thu, May 1, 2014 at 8:33 AM, Eelco Hoogendoorn > <hoogendoorn.eelco(a)gmail.com> wrote: > You problem isn't with colon indexing, but with the > interpretation of the arguments to plot. multiple calls to > plot with scalar arguments do not have the same result as a > single call with array arguments. For this to work as > intended, you would need plt.hold(True), for starters, and > maybe there are other subtleties. > > > On Thu, May 1, 2014 at 1:31 PM, did did <21did21(a)gmx.com> > wrote: > > Hello all and sorry for my bad english, > > i am a beginner with python and i try to save a lot of > data in several folders in a 4D matrix > and then to plot two columns of this 4D matrix. > > Bellow, i have the code to fill my 4D matrix, it works > very well : > > [CODE]matrix4D=[] > for i in Numbers: > readInFolder=folderPrefixe+i+"/" > matrix3D=[] > for j in listeOfdata: > nameOfFile=filePrefixe+i+"-"+j+extensionTXT > nameOfFile=readInFolder+nameOfFile > > matrix2D=np.loadtxt(nameOfFile,delimiter=",",skiprows=1) > matrix3D.append(matrix2D) > matrix4D.append(matrix3D) > array4D = np.asarray(matrix4D)[/CODE] > > But now, i want to plot the third column as function > of the third too (just for trying) and i use > this stupid manner that works well : > > [CODE]plt.figure(1) > temp=plt.plot(array4D[0][0][0][0],array4D[0][0][0][0],'bo') > temp=plt.plot(array4D[0][0][1][0],array4D[0][0][1][0],'bo') > temp=plt.plot(array4D[0][0][2][0],array4D[0][0][2][0],'bo') > temp=plt.plot(array4D[0][0][3][0],array4D[0][0][3][0],'bo') > plt.show()[/CODE] > > Now, i want to use a more smart manner and i use ":" > like this > > [CODE]plt.figure(1) > temp=plt.plot(array4D[0][0][0:3][0],array4D[0][0][0:3][0],'bo') > plt.show()[/CODE] > > The result should be the same but i don't got the same > results!!! > > In attachement you have the two corresponding plots, > can you explain to me with > i don't have the same plots ?? > > thanks for all > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion

1 0

arrays and : behaviour
by did did 01 May '14

01 May '14

Hello all and sorry for my bad english, i am a beginner with python and i try to save a lot of data in several folders in a 4D matrix and then to plot two columns of this 4D matrix. Bellow, i have the code to fill my 4D matrix, it works very well : [CODE]matrix4D=[] for i in Numbers: readInFolder=folderPrefixe+i+"/" matrix3D=[] for j in listeOfdata: nameOfFile=filePrefixe+i+"-"+j+extensionTXT nameOfFile=readInFolder+nameOfFile matrix2D=np.loadtxt(nameOfFile,delimiter=",",skiprows=1) matrix3D.append(matrix2D) matrix4D.append(matrix3D) array4D = np.asarray(matrix4D)[/CODE] But now, i want to plot the third column as function of the third too (just for trying) and i use this stupid manner that works well : [CODE]plt.figure(1) temp=plt.plot(array4D[0][0][0][0],array4D[0][0][0][0],'bo') temp=plt.plot(array4D[0][0][1][0],array4D[0][0][1][0],'bo') temp=plt.plot(array4D[0][0][2][0],array4D[0][0][2][0],'bo') temp=plt.plot(array4D[0][0][3][0],array4D[0][0][3][0],'bo') plt.show()[/CODE] Now, i want to use a more smart manner and i use ":" like this [CODE]plt.figure(1) temp=plt.plot(array4D[0][0][0:3][0],array4D[0][0][0:3][0],'bo') plt.show()[/CODE] The result should be the same but i don't got the same results!!! In attachement you have the two corresponding plots, can you explain to me with i don't have the same plots ?? thanks for all

4 3