Mailman 3 May 2015 - SciPy-Dev

question of scope wavelet code
by Ralf Gommers May 15, 2015

May 15, 2015

Hi all, As you may know, I forked PyWavelets a while ago and with the help of quite a few people started bringing it up to the standards needed for inclusion in scipy: https://github.com/rgommers/pywt. The last year I haven't had too much time for it, but I'd like to get back to it. That's also triggered by several PRs and proposed enhancements - there's quite a bit of interest for a fork that never actually had a release. This proposal https://github.com/rgommers/pywt/issues/58) to add contourlets triggered me to bring up what the scope of a (discrete) wavelets module in scipy should be. What's there now is still mostly what's listed in http://pywavelets.readthedocs.org/en/latest/. A few additions are/were no-brainers, like n-D inverse DWT and inverse SWT (stationary wavelet transform). But beyond that I realize that a bit of care is needed, because there's a huge array of variations on wavelets. Here's a quick list of what I'm aware of, ordered from most to least important in my perspective: - curvelets (https://en.wikipedia.org/wiki/Curvelet) - contourlets (http://en.wikipedia.org/wiki/Contourlet) - shearlets (http://en.wikipedia.org/wiki/Shearlet http://shearlab.org GPL) - chirplets (http://en.wikipedia.org/wiki/Chirplet_transform http://tfd.sourceforge.net/ GPL) - ridgelets ( http://rsta.royalsocietypublishing.org/content/357/1760/2495.short) - wedgelets (http://projecteuclid.org/euclid.aos/1018031261) - noiselets (http://en.wikipedia.org/wiki/Noiselet) - surfacelets (http://dx.doi.org/10.1109/TIP.2007.891785) - bandelets (http://en.wikipedia.org/wiki/Bandelet) At least the first four items in this list seem to be quite widely used. I'd like some opinions on which (if any) of these people think would fit in scipy, and which ones should rather be kept in a wavelets-specific package. Also, if anyone has experience using the contourlet toolbox under discussion in issue 58, would be great to hear how it performed. Ralf

9 11

scipy.optimize enh: modified _differentialevolution.py to enable parallel evaluation of the objective function for the whole population
by Pavel Ponomarev May 13, 2015

May 13, 2015

Hello, Here is the modified file which works for parallel execution of the objective functions for the whole population: https://gist.github.com/pavelponomarev/20ea37324b219c7ed461/revisions The discussion started here: https://github.com/scipy/scipy/issues/4864 The additional boolean parameter "parallel" basically changes the required argument of the objective function from single set of parameters to a list of sets of parameters for the whole population. This enables the execution of DE in parallel. def pareval(listcoords): listresults = joblib.Parallel(n_jobs=num_cores)(delayed(objfunc)(i) for i in listcoords) return listresults result = differential_evolution(pareval, LIMS, parallel=True, polish=False) BR, Pavel

1 0

ANN: Scipy 0.16.0 beta 1 release
by Ralf Gommers May 12, 2015

May 12, 2015

Hi all, I'm pleased to announce the availability of the first beta release of Scipy 0.16.0. Please try this beta and report any issues on the Github issue tracker or on the scipy-dev mailing list. This first beta is a source-only release; binary installers will follow (probably next week). Source tarballs and the full release notes can be found at https://sourceforge.net/projects/scipy/files/scipy/0.16.0b1/. Part of the release notes copied below. Thanks to everyone who contributed to this release! Ralf ========================== SciPy 0.16.0 Release Notes ========================== .. note:: Scipy 0.16.0 is not released yet! SciPy 0.16.0 is the culmination of 6 months of hard work. It contains many new features, numerous bug-fixes, improved test coverage and better documentation. There have been a number of deprecations and API changes in this release, which are documented below. All users are encouraged to upgrade to this release, as there are a large number of bug-fixes and optimizations. Moreover, our development attention will now shift to bug-fix releases on the 0.15.x branch, and on adding new features on the master branch. This release requires Python 2.6, 2.7 or 3.2-3.4 and NumPy 1.6.2 or greater. Highlights of this release include: - A Cython API for BLAS/LAPACK in `scipy.linalg` - A new benchmark suite. It's now straightforward to add new benchmarks, and they're routinely included with performance enhancement PRs. - Support for the second order sections (SOS) format in `scipy.signal`. New features ============ Benchmark suite --------------- The benchmark suite has switched to using `Airspeed Velocity <http://spacetelescope.github.io/asv/>`__ for benchmarking. You can run the suite locally via ``python runtests.py --bench``. For more details, see ``benchmarks/README.rst``. `scipy.linalg` improvements --------------------------- A full set of Cython wrappers for BLAS and LAPACK has been added in the modules `scipy.linalg.cython_blas` and `scipy.linalg.cython_lapack`. In Cython, these wrappers can now be cimported from their corresponding modules and used without linking directly against BLAS or LAPACK. The functions `scipy.linalg.qr_delete`, `scipy.linalg.qr_insert` and `scipy.linalg.qr_update` for updating QR decompositions were added. The function `scipy.linalg.solve_circulant` solves a linear system with a circulant coefficient matrix. The function `scipy.linalg.invpascal` computes the inverse of a Pascal matrix. The function `scipy.linalg.solve_toeplitz`, a Levinson-Durbin Toeplitz solver, was added. Added wrapper for potentially useful LAPACK function ``*lasd4``. It computes the square root of the i-th updated eigenvalue of a positive symmetric rank-one modification to a positive diagonal matrix. See its LAPACK documentation and unit tests for it to get more info. Added two extra wrappers for LAPACK least-square solvers. Namely, they are ``*gelsd`` and ``*gelsy``. Wrappers for the LAPACK ``*lange`` functions, which calculate various matrix norms, were added. Wrappers for ``*gtsv`` and ``*ptsv``, which solve ``A*X = B`` for tri-diagonal matrix ``A``, were added. `scipy.signal` improvements --------------------------- Support for second order sections (SOS) as a format for IIR filters was added. The new functions are: * `scipy.signal.sosfilt` * `scipy.signal.sosfilt_zi`, * `scipy.signal.sos2tf` * `scipy.signal.sos2zpk` * `scipy.signal.tf2sos` * `scipy.signal.zpk2sos`. Additionally, the filter design functions `iirdesign`, `iirfilter`, `butter`, `cheby1`, `cheby2`, `ellip`, and `bessel` can return the filter in the SOS format. The function `scipy.signal.place_poles`, which provides two methods to place poles for linear systems, was added. The option to use Gustafsson's method for choosing the initial conditions of the forward and backward passes was added to `scipy.signal.filtfilt`. New classes ``TransferFunction``, ``StateSpace`` and ``ZerosPolesGain`` were added. These classes are now returned when instantiating `scipy.signal.lti`. Conversion between those classes can be done explicitly now. An exponential (Poisson) window was added as `scipy.signal.exponential`, and a Tukey window was added as `scipy.signal.tukey`. The function for computing digital filter group delay was added as `scipy.signal.group_delay`. The functionality for spectral analysis and spectral density estimation has been significantly improved: `scipy.signal.welch` became ~8x faster and the functions `scipy.signal.spectrogram`, `scipy.signal.coherence` and `scipy.signal.csd` (cross-spectral density) were added. `scipy.signal.lsim` was rewritten - all known issues are fixed, so this function can now be used instead of ``lsim2``; ``lsim`` is orders of magnitude faster than ``lsim2`` in most cases. `scipy.sparse` improvements --------------------------- The function `scipy.sparse.norm`, which computes sparse matrix norms, was added. The function `scipy.sparse.random`, which allows to draw random variates from an arbitrary distribution, was added. `scipy.spatial` improvements ---------------------------- `scipy.spatial.cKDTree` has seen a major rewrite, which improved the performance of the ``query`` method significantly, added support for parallel queries, pickling, and options that affect the tree layout. See pull request 4374 for more details. The function `scipy.spatial.procrustes` for Procrustes analysis (statistical shape analysis) was added. `scipy.stats` improvements -------------------------- The Wishart distribution and its inverse have been added, as `scipy.stats.wishart` and `scipy.stats.invwishart`. The Exponentially Modified Normal distribution has been added as `scipy.stats.exponnorm`. The Generalized Normal distribution has been added as `scipy.stats.gennorm`. All distributions now contain a ``random_state`` property and allow specifying a specific ``numpy.random.RandomState`` random number generator when generating random variates. Many statistical tests and other `scipy.stats` functions that have multiple return values now return ``namedtuples``. See pull request 4709 for details. `scipy.optimize` improvements ----------------------------- A new derivative-free method DF-SANE has been added to the nonlinear equation system solving function `scipy.optimize.root`. Deprecated features =================== ``scipy.stats.pdf_fromgamma`` is deprecated. This function was undocumented, untested and rarely used. Statsmodels provides equivalent functionality with ``statsmodels.distributions.ExpandedNormal``. ``scipy.stats.fastsort`` is deprecated. This function is unnecessary, ``numpy.argsort`` can be used instead. ``scipy.stats.signaltonoise`` and ``scipy.stats.mstats.signaltonoise`` are deprecated. These functions did not belong in ``scipy.stats`` and are rarely used. See issue #609 for details. ``scipy.stats.histogram2`` is deprecated. This function is unnecessary, ``numpy.histogram2d`` can be used instead. Backwards incompatible changes ============================== The deprecated global optimizer ``scipy.optimize.anneal`` was removed. The following deprecated modules have been removed: ``scipy.lib.blas``, ``scipy.lib.lapack``, ``scipy.linalg.cblas``, ``scipy.linalg.fblas``, ``scipy.linalg.clapack``, ``scipy.linalg.flapack``. They had been deprecated since Scipy 0.12.0, the functionality should be accessed as `scipy.linalg.blas` and `scipy.linalg.lapack`. The deprecated function ``scipy.special.all_mat`` has been removed. The deprecated functions ``fprob``, ``ksprob``, ``zprob``, ``randwcdf`` and ``randwppf`` have been removed from `scipy.stats`. Other changes ============= The version numbering for development builds has been updated to comply with PEP 440. Building with ``python setup.py develop`` is now supported. Authors ======= * @axiru + * @endolith * Elliott Sales de Andrade + * Anne Archibald * Yoshiki Vázquez Baeza + * Sylvain Bellemare * Felix Berkenkamp + * Raoul Bourquin + * Matthew Brett * Per Brodtkorb * Christian Brueffer * Lars Buitinck * Evgeni Burovski * Steven Byrnes * CJ Carey * George Castillo + * Alex Conley + * Liam Damewood + * Rupak Das + * Abraham Escalante + * Matthias Feurer + * Eric Firing + * Clark Fitzgerald * Chad Fulton * André Gaul * Andreea Georgescu + * Christoph Gohlke * Andrey Golovizin + * Ralf Gommers * J.J. Green + * Alex Griffing * Alexander Grigorievskiy + * Hans Moritz Gunther + * Jonas Hahnfeld + * Charles Harris * Ian Henriksen * Andreas Hilboll * Åsmund Hjulstad + * Jan Schlüter + * Janko Slavič + * Daniel Jensen + * Johannes Ballé + * Terry Jones + * Amato Kasahara + * Eric Larson * Denis Laxalde * Antony Lee * Gregory R. Lee * Perry Lee + * Loïc Estève * Martin Manns + * Eric Martin + * Matěj Kocián + * Andreas Mayer + * Nikolay Mayorov + * Robert McGibbon + * Sturla Molden * Nicola Montecchio + * Eric Moore * Jamie Morton + * Nikolas Moya + * Maniteja Nandana + * Andrew Nelson * Joel Nothman * Aldrian Obaja * Regina Ongowarsito + * Paul Ortyl + * Pedro López-Adeva Fernández-Layos + * Stefan Peterson + * Irvin Probst + * Eric Quintero + * John David Reaver + * Juha Remes + * Thomas Robitaille * Clancy Rowley + * Tobias Schmidt + * Skipper Seabold * Aman Singh + * Eric Soroos * Valentine Svensson + * Julian Taylor * Aman Thakral + * Helmut Toplitzer + * Fukumu Tsutsumi + * Anastasiia Tsyplia + * Jacob Vanderplas * Pauli Virtanen * Matteo Visconti + * Warren Weckesser * Florian Wilhelm + * Nathan Woods * Haochen Wu + * Daan Wynen + A total of 93 people contributed to this release. People with a "+" by their names contributed a patch for the first time. This list of names is automatically generated, and may not be fully complete.

1 0

adding cut tree function to scipy.cluster
by Skipper Seabold May 12, 2015

May 12, 2015

Hi, I wanted to solicit feedback on adding a function similar to R's cutree [1] to scipy.cluster. PR here [2] It takes a linkage matrix and returns an array where each step is the group to which each observation was merged at any given agglomeration step. Example from the docstring: >>> from scipy import cluster >>> np.random.seed(23) >>> X = np.random.randn(50, 4) >>> Z = cluster.hierarchy.ward(X) >>> cutree = cluster.hierarchy.cut_tree(Z, n_clusters=[5, 10]) >>> cutree[:10] array([[0, 0], [1, 1], [2, 2], [3, 3], [3, 4], [2, 2], [0, 0], [1, 5], [3, 6], [4, 7]]) [1] https://stat.ethz.ch/R-manual/R-patched/library/stats/html/cutree.html [2] https://github.com/scipy/scipy/pull/4796

1 0

Compiler errors and segfaults with current scipy.
by Charles R Harris May 8, 2015

May 8, 2015

I'm seeing things like scipy/sparse/sparsetools/csr.cxx: In function âvoid csr_eliminate_zeros(I, I, I*, I*, T*) [with I = long int; T = long long int]â: scipy/sparse/sparsetools/csr.cxx:9:1: internal compiler error: Segmentation fault This is Redhat g++ 1.9.2. Anyone else seeing problems like this? Chuck

2 1

Re: [SciPy-Dev] dtype int16
by Johnny Chan May 7, 2015

May 7, 2015

Thank you Alex and Sturla for replying. Mini-summary: the tutorial is technically correct (no correction is required). It turns out that there are multiple ways to do this - below list the 3 ways of doing it. (I got the error message the first time due to a syntax error on my part - apologies). Thank you all. I believe it is ok to close off this topic now :) Johnny. p.s. see below email trial for the 3 methods of performing the same operation. On Thu, May 7, 2015 at 3:54 PM, Johnny Chan <johnnychan0302(a)gmail.com> wrote: > Thank you Alex! You are so right! > > I have just tried all 3 methods and everything works and makes sense to me > now. Thank you so much! > > > from numpy import * > > ones( (2,3,4), dtype=int16 ) > Out[2]: > array([[[1, 1, 1, 1], > [1, 1, 1, 1], > [1, 1, 1, 1]], > > [[1, 1, 1, 1], > [1, 1, 1, 1], > [1, 1, 1, 1]]], dtype=int16) > > Restarting kernel... > > > import numpy as np > > np.ones( (2,3,4), dtype=np.int16 ) > Out[2]: > array([[[1, 1, 1, 1], > [1, 1, 1, 1], > [1, 1, 1, 1]], > > [[1, 1, 1, 1], > [1, 1, 1, 1], > [1, 1, 1, 1]]], dtype=int16) > > np.ones( (2,3,4), dtype="int16" ) > Out[3]: > array([[[1, 1, 1, 1], > [1, 1, 1, 1], > [1, 1, 1, 1]], > > [[1, 1, 1, 1], > [1, 1, 1, 1], > [1, 1, 1, 1]]], dtype=int16) > > > Best regards, > Johnny > > > On Thu, May 7, 2015 at 3:47 PM, Alexander Griffing <argriffi(a)ncsu.edu> > wrote: > >> The tutorial is not technically wrong, because it has >> from numpy import * >> >> This lets you write >> ones( (2,3,4), dtype=int16 ) >> as in the tutorial, instead of requiring >> np.ones( (2,3,4), dtype=np.int16 ) >> or alternatively >> np.ones( (2,3,4), dtype="int16" ) >> >> It's confusing that there are so many ways to write this. It doesn't >> follow the principle of https://wiki.python.org/moin/TOOWTDI. >> >> Cheers, >> Alex >> > >

1 0

(no subject)
by Johnny Chan May 7, 2015

May 7, 2015

Hello SciPy-Dev community, Regarding the Tentative NumPy Tutorial (Array Creation) <http://wiki.scipy.org/Tentative_NumPy_Tutorial#head-d3f8e5fe9b903f3c3b2a5c0…>, I have noticed that there might have been a potential typo in one of the code snippet. i.e. I believe... ``` np.ones( (2,3,4), dtype = int16) ``` should be corrected to: ``` np.ones( (2,3,4), dtype = "int16") ``` (Note the double quotes around the int16. Here is my console log to illustrate this: ``` np.ones( (2,3,4), dtype = int16) --------------------------------------------------------------------------- NameError Traceback (most recent call last) <ipython-input-10-6606a1d2e14b> in <module>() ----> 1 np.ones( (2,3,4), dtype = int16) NameError: name 'int16' is not defined np.ones( (2,3,4), dtype = "int16") Out[11]: array([[[1, 1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1]], [[1, 1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1]]], dtype=int16) ``` If this makes sense please could we correct the tutorial documentation? Thanks! Johnny

2 1