Mailman 3 August 2014 - SciPy-Dev

Re: [SciPy-Dev] removing umfpack wrapper
by Robert Cimrman 24 Nov '14

24 Nov '14

On 11/12/2013 11:22 PM, Ralf Gommers wrote: > On Tue, Nov 12, 2013 at 10:20 AM, Robert Cimrman <cimrman3(a)ntc.zcu.cz>wrote: > >> Hi Ralf, >> >> On 11/10/2013 04:36 PM, Ralf Gommers wrote: >>> Hi all, >>> >>> This is long overdue, but I think it's time to remove the UMFPACK wrapper >>> before the next release. I'd like some feedback on how to do that and to >>> what package (if any) to point existing users. >>> >>> As for why to remove the wrapper, see: >>> https://github.com/scipy/scipy/issues/3002 >>> http://permalink.gmane.org/gmane.comp.python.scientific.user/20451 >>> Short summary: UMFPACK changed from a BSD-compatible license to GPL at >> some >>> point. >>> >>> The deprecation warning in sparse.linalg has been referring people to >>> scikits.umfpack until now, however that package has disappeared >> completely >>> as far as I can tell. I suspect it was in the old scikits svn repo and >> was >>> never moved before that was killed. The alternatives seems to be >> Pysparse ( >> >> I missed that the scikit was lost in transition... >> >> How much work would be to move it to the new site - move from SVN to git, >> and? >> I would be willing to do it, if there is interest. >> > > Hi Robert, I think the moving wouldn't be a lot of work (assuming svn > access can still be arranged). If you'd revive the scikit it's not a > one-time effort though - it's only useful if the code is being maintained > imho. That can still be low effort perhaps, but with reviewing some PRs, > maintenance releases, putting it on pypi, etc. I'd expect at least an hour > a week or so. > > Are there scikits.umfpack users now? Would it make sense to salvage > whatever is useful from the scikit and contribute it to pysparse instead? We are using it in sfepy as the default direct solver. It seems to me that the pysparse interface requires the input matrix to be in the LL (linked list) format, which is unfortunate for us, as we use CSR. The scipy (and former scikit) umfpack wrappers have used CSR, so no copies were necessary. (After all, I have created the original scipy wrappers to be used from sfepy in the first place...) So in case others are not interested in having the scikit, I can see two possible solutions from my perspective: either enhance pysparse interface to allow CSR as well, or move the wrappers to sfepy (which I maintain anyway, but swig -> cython conversion would be needed). Not sure yet which solution I prefer. Cheers, r. > Cheers, > Ralf >> >> Cheers, >> r. >> >>> http://pysparse.sourceforge.net) and PyUBlasExt ( >>> https://pypi.python.org/pypi/PyUblasExt). Scikits.sparse is dead(-ish) >> and >>> doesn't wrap umfpack it looks like. PyUBlasExt has something that looks >>> like a wrapper but for C++ (?). So can we refer people to pysparse? >>> >>> Other question: do we call Pysparse instead of scikits.umfpack when >>> available? And if so, by default or not? Or do we rip out everything that >>> sits under useUmfpack=True? >>> >>> Cheers, >>> Ralf

2 5

Linalg: function for diagonalize hermitian commuting matrices simultaneously
by Lars-Hendrik Frahm 07 Oct '14

07 Oct '14

Dear SciPy Developers, I like to contribute a function for simultaneously diagonalize a set of hermitian commuting matrices. Especially for matrices with degenerate eigenvalues. Is there need for something like that in SciPy ? I think it could be helpful for someone else and I didn't find it in recent SciPy versions or on the mailing list. Hope to hear from you.

3 3

Bootstrap confidence limits code
by Constantine Evans 05 Oct '14

05 Oct '14

Hello everyone, A few years ago I implemented a scikit for bootstrap confidence limits (https://github.com/cgevans/scikits-bootstrap). I didn’t think much about it after that until recently, when I realized that some people are actually using it, and that there’s apparently been some talk about implementing this functionality in either scipy.stats or statsmodels (I should thank Randal Olson for discussing this and bringing it to my attention). As such I’ve rewritten most of the code, and written up some docstrings. The current code can do confidence intervals with basic percentile interval, bias-corrected accelerated, and approximate bootstrap confidence methods, and can also provide bootstrap and jackknife indexes. Most of it is implemented from the descriptions in Efron and Tibshirani’s Introduction to the Bootstrap, but the ABC code at the moment is a port from the modified-BSD-licensed bootstrap package for R (not the boot package) as I’m not entirely confident in my understanding of the method. And so, I have a few questions for everyone: * Is there any interest in including this sort of code in either scipy.stats or statsmodels? If so, where do people think would be the better place? The code is relatively small; at the moment it is less than 200 lines, with docstrings probably making up 100 of those lines. * Also, if so, what would need to be changed, added, and improved beyond what is mentioned in the Contributing to Scipy part of the reference guide? I’m never a fan of my own code, and imagine quite a bit would need to be fixed; I know tests will need to be added too. In addition, I have a few questions about what would be better practice for the API, and I haven’t really found a guide on best practices for Scipy: * When I started writing the code, I wrote a single function ci for confidence intervals, with a method argument to choose the method. This is easy for users, especially so that they don’t have to look through documentation to realize that BCA is the most generally useful method (at least from everything I’ve read) and that there really isn’t any reason to use many of the simpler methods. However, ABC takes different paramenters, and needs a statistic function that takes weights, which makes this single-function organization trickier. At the moment, I have a separate function for ABC. Would it be better to split up all the methods to their own functions? * ABC requires a statistic function that takes weights. I’ve noticed that things like np.average takes a weights= argument. Would it be better to require input of a stat(data,weights) function, or input of a stat(data,weights=) with weights as a named argument? The latter would be nice in terms of allowing the same function to be used for all methods, but would make it impossible to use a lambda for the function. Is there some other method of doing this entirely? * Are there any missing features that anyone thinks should be added? I apologize if much of this is answered elsewhere, I just haven’t found any of it; I also apologize if this is far too long-winded and confusing! Regards, Constantine Evans

4 3

last call for numpy 1.8.2 bugfixes
by Julian Taylor 01 Oct '14

01 Oct '14

hi, as numpy 1.9 is going to be a relative hard upgrade as indexing changes expose a couple bugs in third party packages and the large amount of small little incompatibilities I will create a numpy 1.8.2 release tomorrow with a couple of important or hard to work around bugfixes. The most important bugfix is fixing the wrong result partition with multiple selections could produce if selections ended up in an equal range, see https://github.com/numpy/numpy/issues/4836 (if the crash is still unreproducable, help appreciated). the rest of the fixes are small ones listed below. If I have missed one or you consider one of the fixes to invasive for a bugfix release please speak up now. As the number of fixes is small I will skip a release candidate. Make fftpack._raw_fft threadsafe https://github.com/numpy/numpy/issues/4656 Prevent division by zero https://github.com/numpy/numpy/issues/650 Fix lack of NULL check in array_richcompare https://github.com/numpy/numpy/issues/4613 incorrect argument order to _copyto in in np.nanmax, np.nanmin https://github.com/numpy/numpy/issues/4628 Hold GIL for types with fields, fixes https://github.com/numpy/numpy/issues/4642 svd ufunc typo https://github.com/numpy/numpy/issues/4733 check alignment of strides for byteswap https://github.com/numpy/numpy/issues/4774 add missing elementsize alignment check for simd reductions https://github.com/numpy/numpy/issues/4853 ifort has issues with optimization flag /O2 https://github.com/numpy/numpy/issues/4602

6 9

nested setup.py scripts
by Moritz Beber 30 Aug '14

30 Aug '14

Dear all, I want to generate a package with a submodule structure similar to what numpy and scipy use. (Or do you recommend not doing that?) I have read the following pieces of documentation but I'm still unclear about how the main setup.py script discovers the nested scripts and gets the configuration values from those. Is this documented somewhere or can anyone point me to how this is done? Thank you in advance, Moritz P.S.: What I've read: https://github.com/numpy/numpy/blob/master/doc/DISTUTILS.rst.txt http://docs.scipy.org/doc/scipy-dev/reference/hacking.html http://docs.scipy.org/doc/scipy/reference/api.html

3 3

Re: [SciPy-Dev] ANN: NumPy 1.9.0 release candidate 1 available
by Neal Becker 29 Aug '14

29 Aug '14

OK, it's fixed by doing: rm -rf ~/.local/lib/python2.7/site-packages/numpy* python setup.py install --user I guess something was not cleaned out from previous packages

2 1

Nyquist Filters
by Jeff Grasty 29 Aug '14

29 Aug '14

Hi, One of the features that I have found missing in SciPy are functions to design nyquist and root-nyquist filters, such as raised cosine and root-raised cosine filters. I have written several functions for this purpose and was curious if anyone thought was a greater need for this. Thanks, Jeff

3 4

ANN: NumPy 1.9.0 release candidate 1 available
by Julian Taylor 28 Aug '14

28 Aug '14

Hello, Almost punctually for EuroScipy we have finally managed to release the first release candidate of NumPy 1.9. We intend to only fix bugs until the final release which we plan to do in the next 1-2 weeks. In this release numerous performance improvements have been added, most significantly the indexing code has been rewritten be several times faster for most cases and performance of using small arrays and scalars has almost doubled. Plenty of other functions have been improved too, nonzero, where, count_nonzero, floating point min/max, boolean argmin/argmax, searchsorted, triu/tril, masked sorting can be expected to perform significantly better in many cases. Also NumPy now releases the GIL for more functions, most notably the indexing now releases it and the random modules state object has a private lock instead of using the GIL. This allows leveraging pure python threads more efficiently. In order to make working with arrays containing NaN values easier nanmedian and nanpercentile have been added which ignore these values. These functions and the regular median and percentile now also support generalized axis arguments that ufuncs already have, these allow reducing along multiple axis in one call. Please see the release notes for all the details. Please also take not of the many small compatibility notes and deprecation in the notes. https://github.com/numpy/numpy/blob/maintenance/1.9.x/doc/release/1.9.0-not… The source tarballs and win32 binaries can be downloaded here: https://sourceforge.net/projects/numpy/files/NumPy/1.9.0rc1 Cheers, Julian Taylor

1 0

To use C code or Cython code?
by Sri Krishna 27 Aug '14

27 Aug '14

Hi, I'm new to the Scipy-Dev mailing list, looking to contribute wherever I can. I was looking through the open issues and saw this issue <https://github.com/scipy/scipy/issues/3184>, regarding a speed-up for the convolve2d function. My confusion arises from the SciPy coding guidelines <https://github.com/scipy/scipy/blob/master/HACKING.rst.txt> which states that using Cython is much preferable to using plain C/C++/Fortran. Would it be desirable then to change the C code of signal/firfilter.c to a Cythonized code? Thanks, Krishna

4 4

Proposal for a new function nanpdist that treats NaNs as missing values
by Moritz Beber 23 Aug '14

23 Aug '14

Dear all, As suggested in this github issue ( https://github.com/scipy/scipy/issues/3870), I would like to discuss the merit of introducing a new function nanpdist into scipy.spatial. I have also brought up the problem in the following previous e-mail ( http://comments.gmane.org/gmane.comp.python.scientific.devel/18956) and on SO ( http://stackoverflow.com/questions/24781461/compute-the-pairwise-distance-i… ). Warren suggested three ways to tackle this problem: 1. Don't change anything--the users should clean up their data! 2. nanpdist 3. Add a keyword argument to pdist that determines how nan should be treated. Clearly, I don't favor the first option since I believe missing values can be important pieces of information, too. I slightly tend towards option two because adding a keyword will further complicate an already very long pdist function. I'm happy to submit a pull request if there is a consensus that something should be done. Best, Moritz

5 9