Mailman 3 March 2014 - SciPy-Dev

Windows wheels using MKL?
by Matthew Brett March 27, 2014

March 27, 2014

Hi, Can I check what is stopping us building official numpy binary wheels for Windows using the Intel Math Kernel Library? * We'd need developer licenses, but those sound like they would be easy to come by * We'd have to add something to the license for the wheel on the lines of the Canopy license [1], derived from the MKL license [2] - is that a problem? Are there other problems for numpy? * I believe we would also need the Intel Fortran compiler when building 64-bit scipy with MSVC. Is that correct? If we have a license, is that a problem? If we did static linking to MKL for numpy and scipy, is there anything stopping us building wheels that would work for XP and above, for 32 and 64 bit? Maybe this is not the ideal solution, but perhaps it's the right thing to do for now? Cheers, Matthew [1] https://www.enthought.com/products/canopy/canopy-license/ [2] http://software.intel.com/en-us/articles/intel-software-development-product…

2 6

Re: [SciPy-Dev] Multivariate ctypes integration PR
by Brian Lee Newsom March 27, 2014

March 27, 2014

Hi, I believe I have corrected most of the issues brought up by Pauli with the multivariate code. I have not merged the python callback machinery as of now, but that can still be done. I ran my c code through indent using the suggested command to fix formatting issues, removed printfs, fixed comments, added some documentation, and check the return value of c_array_from_tuple. I added a few test cases showing the accuracy of solutions and increase in speed. Unfortunately the unusual function signature means that I cannot use built-in c library functions, so I created the test_multivariate.c file that must be compiled to run the tests. I know this can be built into the process, but I was unsure of how to do that, so I currently only run these tests on a linux system. Please let me know what else I can do to get this merged as soon as possible. Thanks, Brian

1 0

ANN: Scipy 0.14.0 beta 1 release
by Ralf Gommers March 26, 2014

March 26, 2014

Hi, I'm pleased to announce the availability of the first beta release of Scipy0.14.0. Please try this beta and report any issues on the scipy-dev mailing list. Source tarballs, binaries and the full release notes can be found at http://sourceforge.net/projects/scipy/files/scipy/0.14.0b1/. Part of the release notes copied below. A big thank you to everyone who contributed to this release! Ralf SciPy 0.14.0 is the culmination of 8 months of hard work. It contains many new features, numerous bug-fixes, improved test coverage and better documentation. There have been a number of deprecations and API changes in this release, which are documented below. All users are encouraged to upgrade to this release, as there are a large number of bug-fixes and optimizations. Moreover, our development attention will now shift to bug-fix releases on the 0.14.x branch, and on adding new features on the master branch. This release requires Python 2.6, 2.7 or 3.2-3.4 and NumPy 1.5.1 or greater. New features ============ ``scipy.interpolate`` improvements ---------------------------------- A new wrapper function `scipy.interpolate.interpn` for interpolation on regular grids has been added. `interpn` supports linear and nearest-neighbor interpolation in arbitrary dimensions and spline interpolation in two dimensions. Faster implementations of piecewise polynomials in power and Bernstein polynomial bases have been added as `scipy.interpolate.PPoly` and `scipy.interpolate.BPoly`. New users should use these in favor of `scipy.interpolate.PiecewisePolynomial`. `scipy.interpolate.interp1d` now accepts non-monotonic inputs and sorts them. If performance is critical, sorting can be turned off by using the new ``assume_sorted`` keyword. Functionality for evaluation of bivariate spline derivatives in ``scipy.interpolate`` has been added. The new class `scipy.interpolate.Akima1DInterpolator` implements the piecewise cubic polynomial interpolation scheme devised by H. Akima. Functionality for fast interpolation on regular, unevenly spaced grids in arbitrary dimensions has been added as `scipy.interpolate.RegularGridInterpolator` . ``scipy.linalg`` improvements ----------------------------- The new function `scipy.linalg.dft` computes the matrix of the discrete Fourier transform. A condition number estimation function for matrix exponential, `scipy.linalg.expm_cond`, has been added. ``scipy.optimize`` improvements ------------------------------- A set of benchmarks for optimize, which can be run with ``optimize.bench()``, has been added. `scipy.optimize.curve_fit` now has more controllable error estimation via the ``absolute_sigma`` keyword. Support for passing custom minimization methods to ``optimize.minimize()`` and ``optimize.minimize_scalar()`` has been added, currently useful especially for combining ``optimize.basinhopping()`` with custom local optimizer routines. ``scipy.stats`` improvements ---------------------------- A new class `scipy.stats.multivariate_normal` with functionality for multivariate normal random variables has been added. A lot of work on the ``scipy.stats`` distribution framework has been done. Moment calculations (skew and kurtosis mainly) are fixed and verified, all examples are now runnable, and many small accuracy and performance improvements for individual distributions were merged. The new function `scipy.stats.anderson_ksamp` computes the k-sample Anderson-Darling test for the null hypothesis that k samples come from the same parent population. ``scipy.signal`` improvements ----------------------------- ``scipy.signal.iirfilter`` and related functions to design Butterworth, Chebyshev, elliptical and Bessel IIR filters now all use pole-zero ("zpk") format internally instead of using transformations to numerator/denominator format. The accuracy of the produced filters, especially high-order ones, is improved significantly as a result. The new function `scipy.signal.vectorstrength` computes the vector strength, a measure of phase synchrony, of a set of events. ``scipy.special`` improvements ------------------------------ The functions `scipy.special.boxcox` and `scipy.special.boxcox1p`, which compute the Box-Cox transformation, have been added. ``scipy.sparse`` improvements ----------------------------- - Significant performance improvement in CSR, CSC, and DOK indexing speed. - When using Numpy >= 1.9 (to be released in MM 2014), sparse matrices function correctly when given to arguments of ``np.dot``, ``np.multiply`` and other ufuncs. With earlier Numpy and Scipy versions, the results of such operations are undefined and usually unexpected. - Sparse matrices are no longer limited to ``2^31`` nonzero elements. They automatically switch to using 64-bit index data type for matrices containing more elements. User code written assuming the sparse matrices use int32 as the index data type will continue to work, except for such large matrices. Code dealing with larger matrices needs to accept either int32 or int64 indices. Deprecated features =================== ``anneal`` ---------- The global minimization function `scipy.optimize.anneal` is deprecated. All users should use the `scipy.optimize.basinhopping` function instead. ``scipy.stats`` --------------- ``randwcdf`` and ``randwppf`` functions are deprecated. All users should use distribution-specific ``rvs`` methods instead. Probability calculation aliases ``zprob``, ``fprob`` and ``ksprob`` are deprecated. Use instead the ``sf`` methods of the corresponding distributions or the ``special`` functions directly. ``scipy.interpolate`` --------------------- ``PiecewisePolynomial`` class is deprecated. Backwards incompatible changes ============================== scipy.special.lpmn ------------------ ``lpmn`` no longer accepts complex-valued arguments. A new function ``clpmn`` with uniform complex analytic behavior has been added, and it should be used instead. scipy.sparse.linalg ------------------- Eigenvectors in the case of generalized eigenvalue problem are normalized to unit vectors in 2-norm, rather than following the LAPACK normalization convention. The deprecated UMFPACK wrapper in ``scipy.sparse.linalg`` has been removed due to license and install issues. If available, ``scikits.umfpack`` is still used transparently in the ``spsolve`` and ``factorized`` functions. Otherwise, SuperLU is used instead in these functions. scipy.stats ----------- The deprecated functions ``glm``, ``oneway`` and ``cmedian`` have been removed from ``scipy.stats``. ``stats.scoreatpercentile`` now returns an array instead of a list of percentiles. scipy.interpolate ----------------- The API for computing derivatives of a monotone piecewise interpolation has changed: if `p` is a ``PchipInterpolator`` object, `p.derivative(der)` returns a callable object representing the derivative of `p`. For in-place derivatives use the second argument of the `__call__` method: `p(0.1, der=2)` evaluates the second derivative of `p` at `x=0.1`. The method `p.derivatives` has been removed. Authors ======= * Marc Abramowitz + * andbo + * Vincent Arel-Bundock + * Petr Baudis + * Max Bolingbroke * François Boulogne * Matthew Brett * Lars Buitinck * Evgeni Burovski * CJ Carey + * Thomas A Caswell + * Pawel Chojnacki + * Phillip Cloud + * Stefano Costa + * David Cournapeau * Dapid + * Matthieu Dartiailh + * Christoph Deil + * Jörg Dietrich + * endolith * Francisco de la Peña + * Ben FrantzDale + * Jim Garrison + * André Gaul * Christoph Gohlke * Ralf Gommers * Robert David Grant * Alex Griffing * Blake Griffith * Yaroslav Halchenko * Andreas Hilboll * Kat Huang * Gert-Ludwig Ingold * jamestwebber + * Dorota Jarecka + * Todd Jennings + * Thouis (Ray) Jones * Juan Luis Cano Rodríguez * ktritz + * Jacques Kvam + * Eric Larson + * Justin Lavoie + * Denis Laxalde * Jussi Leinonen + * lemonlaug + * Tim Leslie * Alain Leufroy + * George Lewis + * Max Linke + * Brandon Liu + * Benny Malengier + * Matthias Kümmerer + * Cimarron Mittelsteadt + * Eric Moore * Andrew Nelson + * Niklas Hambüchen + * Joel Nothman + * Clemens Novak * Emanuele Olivetti + * Stefan Otte + * peb + * Josef Perktold * pjwerneck * Andrew Sczesnak + * poolio * Jérôme Roy + * Carl Sandrock + * Shauna + * Fabrice Silva * Daniel B. Smith * Patrick Snape + * Thomas Spura + * Jacob Stevenson * Julian Taylor * Tomas Tomecek * Richard Tsai * Joris Vankerschaver + * Pauli Virtanen * Warren Weckesser A total of 78 people contributed to this release. People with a "+" by their names contributed a patch for the first time. This list of names is automatically generated, and may not be fully complete.

6 9

GSoC - what's next
by Ralf Gommers March 24, 2014

March 24, 2014

Hi all, Just a short update, now that the deadline for submitting GSoC proposals has passed. We received four proposals: 1. Leo Mao, "Numpy: Vector Math Library Integration" 2. Janani Padmanbhan, "SciPy/NumPy- enhancements in scipy.special (hyp2f1, sph_harm) " 3. Ankit Agrawal, "SciPy : Discrete Wavelet Transforms and related algorithms" 4. Richard Tsai, "SciPy: Rewrite and improve cluster package in Cython" In principle we have enough mentors for all these proposals, although it looks like I'll have to chase them a bit to sign up in Melange etc. We're going to have to rank these proposals in the next week or so and communicate our preferences to the PSF organizers. I'll be in touch with all potential mentors about that. The announcement by Google of which students were accepted will follow on April 21. Thanks to the four students who spent a lot of effort on creating solid proposals, and to all who helped them by giving feedback. Looks like it's going to be a productive summer! Cheers, Ralf

1 0

Drop support for Python 3.2?
by Charles R Harris March 24, 2014

March 24, 2014

Hi All, The suggestion has been made the we drop Python 3.2 support in numpy 1.9 and scipy 0.15. The advantage, from my point of view, to supporting Python >= 3.3 is that the u'unicode' syntax is supported in 3.3 and this makes it easier to maintain compatibility with Python 2.6 and 2.7. However, it may be a bit early to make this move, so feedback is welcome. Chuck

1 0

Hankel transforms, again
by Tom Grydeland March 23, 2014

March 23, 2014

Hi developers, This is a repost of a message from December 2008 which gave no useful answers. Since then, I’ve had 4-5 requests for the code from people who had a need for it. It’s not a massive demand, but enough that perhaps you’ll consider my offer again. Since the previous posting, I’ve also included alternative filters thanks to Fan-Nian Kong that are shorter and more accurate when the function makes significant changes in more limited intervals. I’m not including the code (since it is mostly thousands of lines of tables), but I will provide the files to anyone who’s interested. Cheers, Tom —— original message below ——— When I recently needed a Hankel transform I was unable to find one in Scipy. I found one in MATLAB though[1], written by Prof. Brian Borchers at New Mexico Tech. The code is based on a previous FORTRAN implementation by W. L. Anderson [2], and the MATLAB code is not marked with any copyright statements. Hankel transforms of the first and second order can be computed through digital filtering. I have rewritten the code in Python/numpy, complete with tests and acknowledgements of the origins, and my employer has agreed that I can donate the code to Scipy. I believe this should be of interest. Hankel transforms arise often in acoustics and other systems with cylinder symmetry. If you want it, I would like a suggestion where it belongs (with other integral transforms, probably) and how I should shape the tests to make them suitable for inclusion. The tests I currently have use Hankel transforms of five different functions with known analytical transforms and compares the transformed values to the numerically evaluated analytical expressions. Currently plots are generated, but for automated testing I suppose something else would be better. Pointing me at an example is sufficient. [1] http://infohost.nmt.edu/~borchers/hankel.html [2] Anderson, W. L., 1979, Computer Program Numerical Integration of Related Hankel Transforms of Orders 0 and 1 by Adaptive Digital Filtering. Geophysics, 44(7):1287-1305. Best regards, -- Tom Grydeland <Tom.Grydeland(a)(norut.no|gmail.com)>

4 13

PR for bounded linear least squares
by Skipper Seabold March 20, 2014

March 20, 2014

Hi All, There's a PR [1] for adding bounded linear least squares that's close to being able to merge. scipy.optimize.bounded_lstsq If you have any specific comments about the PR, etc. Please speak up. Thanks, Skipper https://github.com/scipy/scipy/pull/3137

4 9

EuroScipy sprint
by Ralf Gommers March 18, 2014

March 18, 2014

Hi, Last year's sprint at EuroScipy was quite succesful (20 people, lots of them make their first contribution to Scipy), so this year we should organize one again. Is anyone who's planning to go to the conference interested in coordinating a sprint on the day after the conference (Aug 31)? If not, then I can do it. Cheers, Ralf

1 1

GSoC 2014 : Discrete Wavelets Transform
by Ankit Agrawal March 18, 2014

March 18, 2014

Hi everyone, I have created a page<https://github.com/scipy/scipy/wiki/GSoC-2014-:-Discrete-Wavelet-Transform>on wiki to list the possible tasks that can go along with the project idea of integrating `pywt` library in scipy.signal and addition of some related algorithms(denoising and compression using wavelets) to scikit-image and scipy.signal. Please feel free to suggest or add any other related task. In the coming 2-3 days, I will go through some papers and the pywt codebase to come up with better estimates of the time in which I can complete those tasks. By the end of coming weekend(16th), I hope to have shortlisted the tasks and the timeline for my GSoC proposal. Thanks. Regards, Ankit Agrawal, Communication and Signal Processing, IIT Bombay.

3 8

Re: [SciPy-Dev] SciPy-Dev Digest, Vol 125, Issue 28
by Matt Newville March 18, 2014

March 18, 2014

> Date: Tue, 18 Mar 2014 09:25:48 +0000 (UTC) > From: Sturla Molden <sturla.molden(a)gmail.com> > Subject: Re: [SciPy-Dev] PR for bounded linear least squares > To: scipy-dev(a)scipy.org > Message-ID: > <331411177416826195.432781sturla.molden-gmail.com(a)news.gmane.org> > Content-Type: text/plain; charset=UTF-8 > > Skipper Seabold <jsseabold(a)gmail.com> wrote: > >> I guess I have a question. Does anyone have an option on the best way >> forward for the Fortran 90 code? We have to stick to Fortran 77 code, >> unfortunately, so my options are to either fix the code incrementally >> for F77 or ... ? > > It is basically undoable. Do you know Fortran 77 that well? > > I learned Fortran 90 about 20 years ago. I couldn't write valid Fortran 77 > even if I wanted to ? even though I am fluent in Fortran 90. And that is > probably the case for most scientists that use Fortran today. I would disagree with this but "probably most" is vague enough to be irrefutable. Anyway, Fortran77 is a perfectly reasonable if limited language, and a subset of Fortran90... I'm not sure I completely understood the Skipper's question, but this answer is full of many strange statements. > If you tried, you would even have to change loops into gotos. Good luck on > that... Huh? Fortran77 has do loops. It does not have while loops (though most extensions did, and gfortran does). In any event, a while loop as a goto statement isn't that hard to figure out. The problem with gotos is not the goto statement itself, but the "continue" that is the destination... as one could potentially be coming from anywhere. But if you're writing them yourself and are simulating a while loop, it's no problem. OTOH, Computed gotos are bit more challenging, and a horrible idea. > Coding Fortran 77 in 2014 is not worth the effort. > >> What do people think about using f2c to generate and >> include C code? I've never done this before, and I assume it comes >> with its own can of worms. > > This will not help you out. > > f2c is a Fortran 77 compiler. f2c is a translator of Fortran77 source code into C source code, with an associated library to provide the equivalent of builtin Fortran functions. I believe Skipper was suggesting using this to translate the Fortran into C and then use and support that C code. This is not impossible (I've done it on occasion), but the generated C code is a bit ugly and not quite maintainer-friendly. If it's a one time operation and the Fortran77 code is not too awful, this is not that bad to do really, and the C code can be cleaned up to be readable. Obviously, one needs to be careful with loops and indexing, but it is usually the data types and the use of intrinsic functions (in that Fortran77 overloads intrinsic functions based on types, so that 'sqrt' can take float or double, for example) that are the bigger headaches. Common blocks can be a bit painful too... > f2c and g77 was written by the same person (as one-man projects), and they > basically do the same except for what they emit (C or machine code, > respectively). I believe this is not true. Citation please? My recollection is that f2c came out of Bell Labs, preceding g77 (and written by someone else) by several years. G77 does not emit C but was a frontend for the GCC system, and eventually replaced by gfortran. > The original f77 compiler was a shell script that invoked > f2c and system cc. Huh? There may have been a shell script called f77 (perhaps invoking f2c or g77) on some linux systems, but there have been very many Fortran77 compilers that worked as compilers not translators. > g77 came along because pointer aliasing prevented C > compilers from doing certain optimizations (e.g. register allocation). I believe the main motivation was a desire for a free Fortran77 compiler that worked well with GCC and was better than f2c. > With a modern C compiler it is probably better to use f2c than the g77 compiler, > but it will still be inferior to modern Fortran compilers (gfortran, ifort, > absoft). Unless what you want is C code. --Matt

1 0