Mailman 3 September 2019 - SciPy-Dev

Documentation: user survey
by Maja Gwozdz 31 Oct '19

31 Oct '19

Hi everyone, I'm currently working on improving the SciPy documentation during the Google Season of Docs. One of my projects is about a user survey. The basic idea is to get (documentation) feedback from SciPy users and try to improve our docs accordingly. It'd be great if you could take a look at the survey form and say if you have any comments that could help me. You can access it here: https://docs.google.com/forms/d/e/1FAIpQLSeBAO0UFKDZyKpg2XzRslsLJVHU61ugjc1… Also, I'm compiling a list of websites where I could deploy the survey to get a fair amount of responses. If you know any such websites, please let me know. Your help is much appreciated! All the best, Maja Gwozdz

4 7

New Tutorial on Optimize Help
by Christina Lee 08 Oct '19

08 Oct '19

Hi, I'm a SciPy technical writer and am currently rewriting the scipy.optimize tutorial, focusing on `minimize` right now. While I've gotten a grasp of the "how", I still want to explain "why". Why choose one option over another? I could use information from those with more experience. A lot of methods are available. Most problems can have BFGS thrown at them, but I want to explain something for those other cases. Other situations could have features, like constraints or non-differentiability, that lend themselves to a specific method. But the module still has a lot of alternatives. Are they there for academic purposes? Are they the best for some problems? How could someone find that out? For derivatives, users can choose to provide a function or three different types of finite-difference schemes. When is providing a function better than finite-difference derivatives? For Hessians, approximations are sometimes more efficient. How can we know in advance if that's true? Is that ever true for gradients? How do we choose which finite-difference scheme? `3-point` and `cs` (if it is the symmetric approximation I think) have higher-order accuracy, but `cs` uses a point not yet computed. Is `3-point` ever not the way to go? Thanks for your expertise, Christina

7 9

suggestion re. scipy.spatial.transform.Rotation
by Phillip Feldman 28 Sep '19

28 Sep '19

I've noticed that this class supports four of the five common representations of a rotation. The one that's missing is the rotation matrix. I can supply code that converts an arbitrary sequence of Euler rotations into a rotation matrix. It would also be nice to be able to go in the reverse direction, but I haven't yet figured that out. Phillip scipy.spatial.transform.Rotation <https://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.transfor…> *class *scipy.spatial.transform.Rotation(*quat*, *normalized=False*, *copy=True*)[source] <https://github.com/scipy/scipy/blob/v1.3.0/scipy/spatial/transform/rotation…> <https://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.transfor…> Rotation in 3 dimensions. This class provides an interface to initialize from and represent rotations with: - Quaternions - Direction Cosine Matrices - Rotation Vectors - Euler angles

2 2

PR for overlap-add convolution
by Todd 27 Sep '19

27 Sep '19

I have submitted a PR [1] for a new function, "oaconvolve", which is an implementation of the overlap-add algorithm for convolution. This algorithm is much faster than conventional FFT-based convolution when both inputs are large and one is much bigger than the other. I would be interested in getting your feedback. Any improvements on the algorithm would be greatly appreciated, and I am open to any suggestions on any aspect of the PR. My plans were previously discussed on the mailing list [2]. The original request for this feature dates back to 2009 [3]. More details below for those interested: The function supports all arguments fftconvolve supports, including working on an arbitrary subset of axes of multidimensional arrays. It will also automatically fall back on using fftconvolve in situations where using the overlap-add method doesn't apply (such as when the two inputs are the same shape). This implementation works in a vectorized manner. The input arrays are reshaped so the frequency-domain convolution of all overlapping blocks can be calculated in a single step rather than using a loop like most implementations I have seen. This means that it should work efficiently on any FFT backend that supports vectorization, unlike loops that would be slowed down by FFT backends with long plan creation times. Currently only pocketfft is supported by scipy, but there are plans to support additional FFT backends [4]. This approach should just work for any of them. pocketfft has very low plan creation time so this is less of an issue, but it also supports vectorizing multidimensional FFTs so this approach is still highly efficient for pocketfft. Some potential issues: We previously discussed whether this should be its own function or part of fftconvolve. The performance and memory characteristics of this function are significantly different than those of fftconvolve and would really be used in different situations, so the consensus seemed to be that this should be its own function. I would ultimately like "convolve" to be able to use oaconvolve when it is the best choice, but that is an issue for later. There is a "method" argument for choosing whether to automatically fall back on fftconvolve or raise an error. Being able to prevent falling back on fftconvolve is essentially for testing. We need to be able to make sure that oaconvolve is actually being tested and not the fftconvolve fallback. But this may not be the best way to do that. I have abstracted much of the shared functionality between oaconvolve and fftconvolve to several hidden functions, which should make it easier to add shared features to both. But if this is considered too complicated I can make the functions independent. The tests are slow. There are many potential corner cases with particular combinations of sizes so I have a test marked slow to check a large variety of sizes. This means it is disabled in some situations, but it does need to be run at least sometimes. This is only for one dimension though. Checking more this way would likely be prohibitively slow. There is a detailed docstring, but I didn't add anything to the tutorial. Should I? Are there any additional references anyone things I should add? Is the example okay? Is there another one someone thinks would be useful? I included a bugfix that this code depends on. Should I split it into another pull request or is it okay having them together? The bugfix has to be accepted before the new function can be accepted since oaconvolve will not work without it. [1] https://github.com/scipy/scipy/pull/10869 [2] https://mail.python.org/pipermail/scipy-dev/2019-June/023555.html [3] https://github.com/scipy/scipy/issues/1364 [4] https://mail.python.org/pipermail/scipy-dev/2019-May/023506.html

4 6

suggestion: vector spherical harmonics
by Thomas Hilger 27 Sep '19

27 Sep '19

The generalization of spherical harmonics to vector-fields are the vector spherical harmonics [ https://en.wikipedia.org/wiki/Vector_spherical_harmonics ] . I can provide simple code which relies on scipy's (scalar) spherical harmonics implementation and follows the same conventions to give vector spherical harmonics. Could this be a nice-thing-to-have for scipy? A couple of scientific communities (astronomy, geodesy) use them a lot.

3 2

changing scipy.org to just be about SciPy the library
by Ralf Gommers 24 Sep '19

24 Sep '19

Hi all, The https://scipy.org/ website has always been about SciPy the ecosystem not SciPy the project/library, and it's always resulted in confusion. I propose we finally change that. From my review of a recent PR [1] to the scipy.org website repo: """ Perhaps the confusing thing here is that https://scipy.org is about "SciPy the ecosystem" rather than "SciPy the library". That's what we would really like to fix, then this becomes a lot more straightforward. So far I had in mind to have a replacement "ecosystem site" before changing scipy.org, however we could go without that. There really isn't that much content on the current site that we'd have to remove, and some of the things that are there now (like SciPy Stack) have become irrelevant. We could easily say that `scipy.org` will be about SciPy the library, while keeping a page like https://scipy.org/about.html that explains the ecosystem context. """ Besides some news about and doc hosting for NumPy releases, there's not much content about other projects, and those other projects don't seem to care. The "ecosystem" thing really has had its time. "SciPy ecosystem" isn't even a preferred term anymore I think, the "scientific" is seen as too narrow by a lot of people. PyData or NumPy would be better qualifiers (or some new umbrella term). We may not be able to do all the work straight away, but I want to get approval for the decision to make scipy.org about the SciPy project now. Thoughts? Cheers, Ralf [1] https://github.com/scipy/scipy.org/pull/309

4 3

ANN: SfePy 2019.3
by Robert Cimrman 24 Sep '19

24 Sep '19

I am pleased to announce release 2019.3 of SfePy. Description ----------- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method or by the isogeometric analysis (limited support). It is distributed under the new BSD license. Home page: http://sfepy.org Mailing list: https://mail.python.org/mm3/mailman3/lists/sfepy.python.org/ Git (source) repository, issue tracker: https://github.com/sfepy/sfepy Highlights of this release -------------------------- - interface to eigenvalue problem solvers in SLEPc - new Python 3 enabled Timer class and other Python 3 compatibility fixes For full release notes see [1]. Cheers, Robert Cimrman [1] http://docs.sfepy.org/doc/release_notes.html#id1 --- Contributors to this release in alphabetical order: Robert Cimrman Vladimir Lukes

1 0

Improvement to regular grid interpolation
by Edouard Goudenhoofdt 23 Sep '19

23 Sep '19

Dear scipy developers, One could use *scipy.interpolate.RegularGridInterpolator* at different times with different values but the same source grid and target points. Currently this would recompute the indices each time the function is called. I would like to add the possibility to provide the target points at initialisation and the values when calling the object. Best regards, Edouard Goudenhoofdt

4 6

Re: [SciPy-Dev] SciPy-Dev Digest, Vol 191, Issue 15
by rlucas7＠vt.edu 23 Sep '19

23 Sep '19

> On Sep 18, 2019, at 9:45 AM, scipy-dev-request(a)python.org wrote: > > Send SciPy-Dev mailing list submissions to > scipy-dev(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scipy-dev > or, via email, send a message with subject or body 'help' to > scipy-dev-request(a)python.org > > You can reach the person managing the list at > scipy-dev-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of SciPy-Dev digest..." > > > Today's Topics: > > 1. Re: improvement to binned statistic (Ralf Gommers) > 2. Adding alpha complexes/filtrations to scipy.spatial? > (Hamilton, Wesley) > 3. Re: Improvement to regular grid interpolation (Simon S. Clift) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 18 Sep 2019 15:02:17 +0200 > From: Ralf Gommers <ralf.gommers(a)gmail.com> > To: SciPy Developers List <scipy-dev(a)python.org> > Subject: Re: [SciPy-Dev] improvement to binned statistic > Message-ID: > <CABL7CQhHJ-qJmbNnmJeGYATLKZQZCc6z9EB-RivXxKBUo8pscA(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi Edouard, > > > On Wed, Sep 18, 2019 at 11:29 AM Edouard Goudenhoofdt <egouden(a)gmail.com> > wrote: > >> Dear scipy developers, >> >> One could use scipy.stats.binned_statistic_dd for the same sample points >> but for values available at different times. >> Currently this involves the computation of the bin numbers every time the >> function is called. >> Therefore I would like to add an optional argument "binnumbers" to skip >> this step when calling the function again. >> > > That seems sensible. Could you check that creating the bin numbers really > takes the majority of the time? There's also a fair amount of input > validation that shouldn't be skipped even when a new `binnumbers` is passed > in. If that is the case, sending a PR with a benchmark would be very > welcome. > > Cheers, > Ralf IIUC Edouard what you’d like to do is take input data, run binned_statistic_dd() and then do the same thing with the bin edges calculated from this first call either on a new input dataset or on the same data(perhaps calculating on a new statistic?). AFAIK the binned_statistic_dd() function isn’t able to take binedges as an argument. If you want multiple stats for the same data I think you can achieve that via a custom callable() that returns multiple statistics rather than a single scalar, but I haven’t done this so you should confirm that the approach would work fine. If you want to take that up I’m happy to review the PR. If not, and this is something others agree is useful and should be implemented, it seems reasonable to do. I can implement if you don’t have time or are otherwise unable to open a PR. Let me know either way. -Lucas Roberts

2 1

Re: [SciPy-Dev] Adding alpha complexes/filtrations to scipy.spatial?
by Hamilton, Wesley 18 Sep '19

18 Sep '19

These are the same as alpha shapes: alpha complexes are alpha shapes, and an alpha filtration is a collection of alpha shapes corresponding to varying alpha. The situation I've had to use these for is in studying protein structure (which is one of the original applications for this stuff), though I'm sure there are other uses (like for scikit image?). In practice, these constructions can work in any dimension, but geometrically I think the intuition changes drastically above 3 dimensions. In terms of computational complexity, I think the worst bit is in computing determinants for the in-circle radius and center, but again for small dimensions it isn't terrible. I'd also be happy to look into more optimal (if applicable) implementations for this construction, including reaching out to, say, Edelsbrunner and other experts. I can also share the code somewhere, or if anyone wants to discuss via direct email, then that would also be great. I admit I'm relatively new to software development (and git, etc.), but I'm hoping this code can be implemented and useful to others! Best, Wesley

1 0