Mailman 3 April 2019 - SciPy-Dev

GSoC 2019: Project Proposal
by Anany Shrey Jain 15 Apr '19

15 Apr '19

Hey, Thanks Greg for going through my proposal. I appreciate the way you went through every small detail . It will be very difficult for me to make changes in the implementation details now. Moreover the API will be changed a lot after the discussions with other developers, so for now, it will be better to stick with the current API . Except the API I have incorporated all the suggestions. The week 5 was left intentionally as a buffer during the phase 1 evaluations (I admit that I should have mentioned that in my proposal) . Nothing special was planned for mkl_fft, I just thought there might be need of some conversions lsubroutines ike this : https://github.com/pyFFTW/pyFFTW/blob/e9966e5d215a00f89156bd88e1d42b884f85d… . But after seeing the Mkl code base again I don't think it will be required. Luckily, I have that kind of GPU (Cuda 9.1) but still it will be very difficult to reach to a common point and get things merged in CuPy. So I have removed the text about it, during the mentioned week, from my proposal. Here is the link to my proposal: https://docs.google.com/document/d/1TIiH9j3rPAN--53MwjZz7NFO1GGtxmEH3RiGzvq… -Anany Date: Mon, 8 Apr 2019 01:54:00 -0400 > From: Gregory Lee <grlee77(a)gmail.com> > To: SciPy Developers List <scipy-dev(a)python.org> > Subject: Re: [SciPy-Dev] GSoC 2019: Project Proposal > Message-ID: > < > CAJR3sXdxc-3EHBMXHwtT4_5pVM-J2Lb9_FV1y8v1fBs9J3iC3w(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > On Sat, Apr 6, 2019 at 5:08 PM Anany Shrey Jain < > f20170920(a)goa.bits-pilani.ac.in> wrote: > > > > > Hey , > > Thanks Greg for a detailed reply. I will be working on both > > https://github.com/pyFFTW/pyFFTW/pull/256 and > > https://github.com/cupy/cupy/issues/1936 as a part of this project. I > > have constructed my API implementation based on the `set_backend` > function > > approach, but in comparatively cleaner way. I have written my proposal > for > > this project and here is a link to it: > > > https://docs.google.com/document/d/1TIiH9j3rPAN--53MwjZz7NFO1GGtxmEH3RiGzvq… > . > > Please go through my proposal and if possible send some feedback on it. > > > > -Anany > > > > > Hi Anany, > > I appreciate that you have narrowed down some of the possibilities. The API > will need further refinement to reach consensus, but the general concept as > presented seems reasonable and I don't think we have to reach consensus on > the finer implementation details for purposes of the proposal. I am not > sure it is necessary to have the `set_backend` function return a class that > allows setting additional arguments. Ralf's made a comment earlier that was > in favor of just supporting the arguments currently present in SciPy's > fftpack. I was also thinking something a bit simpler in terms of just > reading the name of the backend from a configuration file or environment > variable as opposed to running a full configuration script, but others may > disagree. > > I will comment mainly on the timeline section of the proposal as we did not > discuss that previously: > > I think the timeline is a bit overly optimistic and dedicates too much time > to third party libraries. It is probably not realistically going to be > possible to complete changes to SciPy as well as pyFFTW, mkl_fft and CuPy. > I would focus the majority of the timeline on SciPy itself. You can leave > the text about working on the pyFFTW and CuPy issues as goals for weeks > 9-12, but I think all work in the early weeks should be in SciPy itself and > not require any changes to the third-party libraries in order for the > backend code to function. Each of the third party libraries already has a > substantial portion of the SciPy API implemented and can usefully be > integrated as backends as-is. > > I think you can keep the plan to implement an initial interface and pyFFTW > support in weeks 1-2, but I think week 3-4 should be primarily along the > lines of improving and revising the initial interface based on reviewer > feedback from your first PR. Realistically, it will take a substantial > amount of time to reach agreement among all parties and code will not be > merged right away. I would focus weeks 3-4 on revising based on feedback > rather than proposing to "make CuPy compatible". CuPy already has > compatible interfaces for many of the functions ( > https://github.com/cupy/cupy/blob/master/cupyx/scipy/fftpack/fft.py). > > Week 5 seems to be missing from the timeline? > The week 5 was left intentionally as a buffer during the phase 1 evaluations (I admit that I should have mentioned that in my proposal) . > > I would remove the item about adding changes to mkl_fft (in week 6). Was > there something specific you had in mind for that? Nothing special was planned for mkl_fft, I just thought there might be need of some conversions like this : https://github.com/pyFFTW/pyFFTW/blob/e9966e5d215a00f89156bd88e1d42b884f85d… . But after seeing the Mkl codebase again I don't think it will be required. There are already > existing wrappers ( > https://github.com/IntelPython/mkl_fft/blob/master/mkl_fft/_scipy_fft.py) > and I'm not sure there is much more to do there. A suitable replacement > for the week 5-6 section of the timeline could be working on the SciPy user > documentation detailing how to use and configure the backends. It could be > considered to potentially make small PRs to third party library > documentation as well, mentioning how scipy can be configured to use them > as its default backend. For instance, the new backend-based approach will > be preferable to the monkey patching currently discussed in the pyFFTW > docs. > > I proposed above to remove mention of CuPy from weeks 3-4 to reduce the > overall proportion of time in the proposal that is related to CuPy for a > couple of reasons: > > 1.) There is no existing precedent for interfacing with GPU code from SciPy > so that may be a little controversial among the larger group of SciPy > developers and have a harder time of getting merged. If we assume NumPy > inputs should give NumPy outputs then the required host/device memory > transfers will reduce performance, making the GPU-based approach only > beneficial for large transforms. We could allow GPU-based input/output, but > other functions in SciPy itself do not support GPU-based arrays. > > 2.) Using CuPy requires access to a relatively recent (~last five years or > so) NVidia GPU that supports CUDA (I think at least CUDA 8.0 is required). > Do you have access to an appropriate GPU? Luckily I have that kind of GPU (Cuda 9.1) but still it will be very difficult to reach to a common point and get things merged in CuPy. So I have removed the text about it from my proposal. There is no requirement for > applicants to have access to such hardware, but it will difficult to > propose making contributions to CuPy itself without it. We may be able to > work out remote access to a machine that has appropriate hardware, but I > have not looked into specifics of how we would arrange that yet and cannot > guarantee it. > > > > > Message: 1 > > Date: Fri, 5 Apr 2019 12:13:15 -0400 > > From: Gregory Lee <grlee77(a)gmail.com> > > To: SciPy Developers List <scipy-dev(a)python.org> > > Subject: Re: [SciPy-Dev] GSoC 2019 > > Message-ID: > > < > > CAJR3sXeVPAB+jGB_EdxiDg45jbR4wHzLVjjBk7+RiesT0XMrpw(a)mail.gmail.com> > > Content-Type: text/plain; charset="utf-8" > > > > On Fri, Apr 5, 2019 at 4:25 AM Anany Shrey Jain < > > f20170920(a)goa.bits-pilani.ac.in> wrote: > > > > > Hey Greg, > > > The project "Revamp FFTPACK" also aims at contributing to third party > > > libraries to fill in functionality gaps. For pyFFTW I can see that work > > on > > > adding Real-to-Real transforms has been going on for a long time. So > do I > > > need to continue the work going on in > > > https://github.com/pyFFTW/pyFFTW/pull/256 as a part of this project? > > > > > > > It is not a requirement that you work on the pyFFTW wrappers, but that > > would be a good complement to this project. Here are some thoughts. > > > > 1.) I would first have a mechanism to fall back to the existing > > scipy.fftpack implementations for any functions not currently supported > by > > a given backend. pyFFTW currently already falls back to scipy's own > > implementation for DCT and DST transforms ( > > > > > https://github.com/pyFFTW/pyFFTW/blob/e9966e5d215a00f89156bd88e1d42b884f85d… > > ). > > The PR you referenced would allow it to use FFTW to perform these > > transforms instead and there would be no remaining functions that fall > back > > to scipy.fftpack. > > > > 2.) You can work on adding real-to-real support in supporting libraries, > > but this is a secondary goal to pursue once the SciPy side of the project > > has been completed. > > pyFFTW builds on FFTW which does already have functions providing > > real-to-real transforms. So what you see in the PR referenced above is > > mainly adding the wrappers for those to pyFFTW. > > > > > > > > > What needs to be done for the other two libraries? Do I need to make a > > > similar PR to other two libraries as well? > > > > > > > > You could potentially try that, but I think that is more of a stretch > goal. > > The approach for CuPy or mkl-fft would have to be different, because for > > those two cases the underlying libraries DO NOT have dedicated functions > > implementing the real-to-real transforms. Also, the underlying FFT > > libraries are proprietary, so we cannot propose to add new transforms on > > the C/C++ side. However, it is possible in principle to implement > > real-to-real transforms using only the already existing complex-valued > > transforms (albeit at some additional computational/memory cost). Some > > detailed examples of this are given in the following thread on DSP > > StackExchange: > > > https://dsp.stackexchange.com/questions/2807/fast-cosine-transform-via-fft > > > > I think that is probably the only feasible route for real-to-real > > transforms using those two backends. I think whether or not this type of > > implementation would give a performance benefit over just reusing the > > existing scipy.fftpack implementations is unknown. > > > > One FFT-related issues that could be improved in CuPy in a more > > straightforward manner than adding real-to-real support, is improving the > > real-to-complex and complex-to-real support. These transforms are > > supported by the underlying CUFFT library, but support for this within > CuPy > > is incomplete. One issue related to this was opened at ( > > https://github.com/cupy/cupy/issues/1936), but no PR implementing it has > > been made. > > > > > > > For all the three libraries there are SciPy-compatable routines already > > > defined. After getting required information using the newly created > > > interface I need to use them at the backend, right? > > > > > > > Yes, you should rely on the already existing interfaces when implementing > > the backends in SciPy. Any additional 3rd party libraries would have to > > provide a scipy-like interface in order to be compatible for use as a > > backend. > > > > > > > > > > > > -Anany > > > > > >> ---------------------------------------------------------------------- > > >> > > >> Message: 1 > > >> Date: Thu, 4 Apr 2019 10:58:27 -0400 > > >> From: Gregory Lee <grlee77(a)gmail.com> > > >> To: SciPy Developers List <scipy-dev(a)python.org> > > >> Subject: Re: [SciPy-Dev] SciPy-Dev Digest, Vol 185, Issue 18 > > >> Message-ID: > > >> < > > >> CAJR3sXcJuqQJrDboN5zH-pd7NG-wD8EWCdTCAtQ+vJpnZjKsRw(a)mail.gmail.com> > > >> Content-Type: text/plain; charset="utf-8" > > >> > > >> Hi Anany, > > >> > > >> It is nice to see that you have updated with multiple potential APIs. > > Can > > >> some other core SciPy devs also help give feedback on what SciPy would > > >> find > > >> preferable? Out of the four you proposed, I think something along the > > >> lines > > >> of the second method is cleanest in terms of setting the backend at > > >> runtime. I think having some aspect from your 4th variant where the > > >> default > > >> backend can be read from a configuration file and/or environment > > variable > > >> is also desirable so that the user's desired default can always be > > >> automatically set at import time. > > >> > > >> I disagree with the following specific statement, though: > > >> "It is better to use all the above mentioned ideas because every idea > > has > > >> its own benefits besides it gives user the luxury to choose the method > > >> implementation of backend." > > >> > > >> I do not think we want to provide multiple ways to do the same thing. > It > > >> can be confusing to users and more work for devs to maintain. I think > > >> there > > >> should be one method of setting the backend that the user can either > > call > > >> at run time or have it be automatically called based on some sort of > > >> configuration file and/or environment variable(s) at import time. > > >> > > >> - Greg > > >> > > >> > > >> On Tue, Apr 2, 2019 at 2:19 PM Anany Shrey Jain < > > >> f20170920(a)goa.bits-pilani.ac.in> wrote: > > >> > > >> > Hey, > > >> > I have already sent the link to the page containing the list of > ideas > > >> for > > >> > implementation of new API that I propose for this project. Would you > > >> like > > >> > to add something to it or give some comments on it? The Gsoc > proposal > > >> > deadline is 9th of April and I am planning to put all of this in my > > >> > proposal to make it effective . So any more ideas or comments will > be > > >> > really helpful for me . Here is the link to the page: > > >> > https://github.com/ananyashreyjain/Gsoc-Scipy/wiki/Gsoc-2019:-Scipy > . > > >> > > > >> > On Sun, Mar 31, 2019 at 5:23 AM <scipy-dev-request(a)python.org> > wrote: > > >> > > > >> >> > > >> >> Message: 1 > > >> >> Date: Sat, 30 Mar 2019 13:31:51 -0400 > > >> >> From: Eric Larson <larson.eric.d(a)gmail.com> > > >> >> To: SciPy Developers List <scipy-dev(a)python.org> > > >> >> Subject: Re: [SciPy-Dev] GSoC 2019 > > >> >> Message-ID: > > >> >> < > > >> >> CAGu2niVWsKrMukfWcAWAtj-1NBm2utCeh3BOnNku7jNoAcx-kw(a)mail.gmail.com > > > > >> >> Content-Type: text/plain; charset="utf-8" > > >> >> > > >> >> It looks like the main ideas are in these bits of code, but not > > >> explicitly > > >> >> stated. In brief it looks like the proposed API is to add a > > >> `fft_object` > > >> >> kwarg to every function in `scipy.fftpack`. It would be good to > > discuss > > >> >> this choice explicitly -- what advantages and disadvantages it has > > >> >> relative > > >> >> to other approaches (e.g., creating new namespaces, or having a > > >> >> `set_backend` function (and/or context manager) that would change > > >> things > > >> >> under the hood). > > >> >> > > >> >> If you make changes and need more feedback, let us know. The more > > >> specific > > >> >> you can be about what you need feedback on (high level ideas, low > > level > > >> >> implementation, etc.), the better. > > >> >> > > >> >> Eric > > >> >> > > >> >> > > >> >> On Fri, Mar 29, 2019 at 4:12 AM Anany Shrey Jain < > > >> >> f20170920(a)goa.bits-pilani.ac.in> wrote: > > >> >> > > >> >> > Hey, > > >> >> > I went through the idea list of Scipy for this years's GSoC and I > > >> intend > > >> >> > to work towards the idea of revamping the Scipy's fftpack. I have > > >> >> already > > >> >> > been contributing to Scipy since the end of last year and I have > > some > > >> >> > knowledge about Fourier transforms as it was the part of my > > >> discipline. > > >> >> I > > >> >> > could not take part in this discussion earlier because my mid > > >> semester > > >> >> > exams were going on, but since they are now over so I can devote > as > > >> much > > >> >> > time as required on this project. I have already created a github > > >> page > > >> >> with > > >> >> > some code snippets and functionalities that we can implement, > here > > is > > >> >> the > > >> >> > link to it: > > >> >> > > > https://github.com/ananyashreyjain/Gsoc-Scipy/wiki/Gsoc-2019:-Scipy > > >> . > > >> >> > > > >> >> > Cheers, > > >> >> > Anany Jain > > >> >> > > > >> >> > _______________________________________________ > > >> >> > SciPy-Dev mailing list > > >> >> > SciPy-Dev(a)python.org > > >> >> > https://mail.python.org/mailman/listinfo/scipy-dev > > >> >> > > > >> >>

1 1

Re: [SciPy-Dev] Google Season of Docs
by Christoph Baumgarten 14 Apr '19

14 Apr '19

I created a first draft in a forked repo: https://github.com/chrisb83/gsod/blob/gsod-stats/2019/SciPy_ideas_list.md Feedback is welcome! https://github.com/chrisb83/gsod/pull/1 On Thu, Apr 4, 2019 at 4:43 PM <scipy-dev-request(a)python.org> wrote: > Send SciPy-Dev mailing list submissions to > scipy-dev(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scipy-dev > or, via email, send a message with subject or body 'help' to > scipy-dev-request(a)python.org > > You can reach the person managing the list at > scipy-dev-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of SciPy-Dev digest..." > > > Today's Topics: > > 1. Re: Google Season of Docs (Ralf Gommers) > 2. Re: GSoC 2019 Projects (Gregory Lee) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 4 Apr 2019 13:26:40 +0200 > From: Ralf Gommers <ralf.gommers(a)gmail.com> > To: SciPy Developers List <scipy-dev(a)python.org> > Subject: Re: [SciPy-Dev] Google Season of Docs > Message-ID: > <CABL7CQg5Qyjb= > xf_JMmAOvLamd330m47E1Y356-cLfj99TOXKA(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > On Wed, Apr 3, 2019 at 11:49 PM Paul van Mulbregt < > p.vanmulbregt(a)comcast.net> > wrote: > > > I can be a second mentor for the stats documentation proposal. > > > > Great, thanks Paul! > > > > -Paul > > > > On Apr 3, 2019, at 3:12 PM, Christoph Baumgarten < > > christoph.baumgarten(a)gmail.com> wrote: > > > > > > Sure, I will add it in the next few days. > > > > On Tue, Apr 2, 2019 at 6:02 PM <scipy-dev-request(a)python.org> wrote: > > > >> Send SciPy-Dev mailing list submissions to > >> scipy-dev(a)python.org > >> > >> To subscribe or unsubscribe via the World Wide Web, visit > >> https://mail.python.org/mailman/listinfo/scipy-dev > >> or, via email, send a message with subject or body 'help' to > >> scipy-dev-request(a)python.org > >> > >> You can reach the person managing the list at > >> scipy-dev-owner(a)python.org > >> > >> When replying, please edit your Subject line so it is more specific > >> than "Re: Contents of SciPy-Dev digest..." > >> > >> > >> Today's Topics: > >> > >> 1. Re: Google Season of Docs (Ralf Gommers) > >> > >> > >> ---------------------------------------------------------------------- > >> > >> Message: 1 > >> Date: Tue, 2 Apr 2019 10:40:20 +0200 > >> From: Ralf Gommers <ralf.gommers(a)gmail.com> > >> To: SciPy Developers List <scipy-dev(a)python.org> > >> Subject: Re: [SciPy-Dev] Google Season of Docs > >> Message-ID: > >> <CABL7CQipvttnWrAE= > >> V6Fr+iJKjGM1x0wgKbK-ea7ZNsOXx6YDg(a)mail.gmail.com> > >> Content-Type: text/plain; charset="utf-8" > >> > >> On Fri, Mar 22, 2019 at 8:07 PM Ralf Gommers <ralf.gommers(a)gmail.com> > >> wrote: > >> > >> > > >> > > >> > On Thu, Mar 21, 2019 at 12:10 PM Christoph Baumgarten < > >> > christoph.baumgarten(a)gmail.com> wrote: > >> > > >> >> > >> >> Hi, > >> >> > >> >> taking up Ralf's mail on GSoD, I had a look at the website and > propose > >> to > >> >> apply. > >> >> > >> >> My project suggestion would be to work on the documentation of a > module > >> >> like stats: > >> >> > >> > > >> >> - improve the tutorial site > >> >> https://docs.scipy.org/doc/scipy/reference/tutorial/stats.html > >> >> - improve the documentation of the API > >> >> - check whether current descriptions can be improved > >> >> - add examples and references > >> >> - cross-ref to other relevant / similar functions in the module > >> >> > >> >> The second point would require decent knowledge of statistics whereas > >> the > >> >> first point should be easier to tackle for a technical writer. So I > >> see the > >> >> second point as optional and one could also try to work on the > tutorial > >> >> site of more than one module during the 3-month period. > >> >> > >> > > >> >> The task would leave quite a bit of freedom to the writer to choose > >> what > >> >> areas to focus on, potentially taking up some of the documentation > >> issues > >> >> that are already on Github. > >> >> > >> >> I could assist as a mentor for stats (we need at least two mentors). > >> >> > >> >> Any suggestions or comments? > >> >> > >> > > >> > Thanks Christoph, this sounds like a good idea to me. We have a lot of > >> > content on scipy.stats, and a tech writer could really make a > >> difference in > >> > making that easier to understand for users. I suggest we write this up > >> as > >> > an idea on our (to be created) ideas page as soon as we know whether > we > >> > have an umbrella organization that we can participate with. > >> > > >> > >> An update on this: NumFOCUS is applying as an umbrella org. We'll need > to > >> add our ideas at > >> https://github.com/numfocus/gsod/blob/master/2019/SciPy_ideas_list.md. > >> Since there are going to be few spots available, it's the quality of the > >> content that matters for acceptance of NumFOCUS as an umbrella org. We > >> should have 1-2 ideas that are well described up on that page by April > >> 14th. There are some guidelines at > >> > >> > https://developers.google.com/season-of-docs/docs/project-ideas#project-idea > >> > >> Christoph, could you add your idea in that SciPy_ideas_list.md file? > >> > >> I will add some more general information there about getting started on > >> the > >> SciPy docs. Also, if anyone has another idea, that would be great to > hear. > >> > >> Cheers, > >> Ralf > >>

2 1

plan for SciPy 1.3.0 release
by Ralf Gommers 12 Apr '19

12 Apr '19

Hi all, It's about time to start thinking about the next release. We should probably be aiming for around mid May. Is anyone interested to be the release manager for this release? Cheers, Ralf

4 8

GSoC 2019: Project Proposal
by Anany Shrey Jain 09 Apr '19

09 Apr '19

Hey , Thanks Greg for a detailed reply. I will be working on both https://github.com/pyFFTW/pyFFTW/pull/256 and https://github.com/cupy/cupy/issues/1936 as a part of this project. I have constructed my API implementation based on the `set_backend` function approach, but in comparatively cleaner way. I have written my proposal for this project and here is a link to it: https://docs.google.com/document/d/1TIiH9j3rPAN--53MwjZz7NFO1GGtxmEH3RiGzvq…. Please go through my proposal and if possible send some feedback on it. -Anany Message: 1 > Date: Fri, 5 Apr 2019 12:13:15 -0400 > From: Gregory Lee <grlee77(a)gmail.com> > To: SciPy Developers List <scipy-dev(a)python.org> > Subject: Re: [SciPy-Dev] GSoC 2019 > Message-ID: > < > CAJR3sXeVPAB+jGB_EdxiDg45jbR4wHzLVjjBk7+RiesT0XMrpw(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > On Fri, Apr 5, 2019 at 4:25 AM Anany Shrey Jain < > f20170920(a)goa.bits-pilani.ac.in> wrote: > > > Hey Greg, > > The project "Revamp FFTPACK" also aims at contributing to third party > > libraries to fill in functionality gaps. For pyFFTW I can see that work > on > > adding Real-to-Real transforms has been going on for a long time. So do I > > need to continue the work going on in > > https://github.com/pyFFTW/pyFFTW/pull/256 as a part of this project? > > > > It is not a requirement that you work on the pyFFTW wrappers, but that > would be a good complement to this project. Here are some thoughts. > > 1.) I would first have a mechanism to fall back to the existing > scipy.fftpack implementations for any functions not currently supported by > a given backend. pyFFTW currently already falls back to scipy's own > implementation for DCT and DST transforms ( > > https://github.com/pyFFTW/pyFFTW/blob/e9966e5d215a00f89156bd88e1d42b884f85d… > ). > The PR you referenced would allow it to use FFTW to perform these > transforms instead and there would be no remaining functions that fall back > to scipy.fftpack. > > 2.) You can work on adding real-to-real support in supporting libraries, > but this is a secondary goal to pursue once the SciPy side of the project > has been completed. > pyFFTW builds on FFTW which does already have functions providing > real-to-real transforms. So what you see in the PR referenced above is > mainly adding the wrappers for those to pyFFTW. > > > > > What needs to be done for the other two libraries? Do I need to make a > > similar PR to other two libraries as well? > > > > > You could potentially try that, but I think that is more of a stretch goal. > The approach for CuPy or mkl-fft would have to be different, because for > those two cases the underlying libraries DO NOT have dedicated functions > implementing the real-to-real transforms. Also, the underlying FFT > libraries are proprietary, so we cannot propose to add new transforms on > the C/C++ side. However, it is possible in principle to implement > real-to-real transforms using only the already existing complex-valued > transforms (albeit at some additional computational/memory cost). Some > detailed examples of this are given in the following thread on DSP > StackExchange: > https://dsp.stackexchange.com/questions/2807/fast-cosine-transform-via-fft > > I think that is probably the only feasible route for real-to-real > transforms using those two backends. I think whether or not this type of > implementation would give a performance benefit over just reusing the > existing scipy.fftpack implementations is unknown. > > One FFT-related issues that could be improved in CuPy in a more > straightforward manner than adding real-to-real support, is improving the > real-to-complex and complex-to-real support. These transforms are > supported by the underlying CUFFT library, but support for this within CuPy > is incomplete. One issue related to this was opened at ( > https://github.com/cupy/cupy/issues/1936), but no PR implementing it has > been made. > > > > For all the three libraries there are SciPy-compatable routines already > > defined. After getting required information using the newly created > > interface I need to use them at the backend, right? > > > > Yes, you should rely on the already existing interfaces when implementing > the backends in SciPy. Any additional 3rd party libraries would have to > provide a scipy-like interface in order to be compatible for use as a > backend. > > > > > > > -Anany > > > >> ---------------------------------------------------------------------- > >> > >> Message: 1 > >> Date: Thu, 4 Apr 2019 10:58:27 -0400 > >> From: Gregory Lee <grlee77(a)gmail.com> > >> To: SciPy Developers List <scipy-dev(a)python.org> > >> Subject: Re: [SciPy-Dev] SciPy-Dev Digest, Vol 185, Issue 18 > >> Message-ID: > >> < > >> CAJR3sXcJuqQJrDboN5zH-pd7NG-wD8EWCdTCAtQ+vJpnZjKsRw(a)mail.gmail.com> > >> Content-Type: text/plain; charset="utf-8" > >> > >> Hi Anany, > >> > >> It is nice to see that you have updated with multiple potential APIs. > Can > >> some other core SciPy devs also help give feedback on what SciPy would > >> find > >> preferable? Out of the four you proposed, I think something along the > >> lines > >> of the second method is cleanest in terms of setting the backend at > >> runtime. I think having some aspect from your 4th variant where the > >> default > >> backend can be read from a configuration file and/or environment > variable > >> is also desirable so that the user's desired default can always be > >> automatically set at import time. > >> > >> I disagree with the following specific statement, though: > >> "It is better to use all the above mentioned ideas because every idea > has > >> its own benefits besides it gives user the luxury to choose the method > >> implementation of backend." > >> > >> I do not think we want to provide multiple ways to do the same thing. It > >> can be confusing to users and more work for devs to maintain. I think > >> there > >> should be one method of setting the backend that the user can either > call > >> at run time or have it be automatically called based on some sort of > >> configuration file and/or environment variable(s) at import time. > >> > >> - Greg > >> > >> > >> On Tue, Apr 2, 2019 at 2:19 PM Anany Shrey Jain < > >> f20170920(a)goa.bits-pilani.ac.in> wrote: > >> > >> > Hey, > >> > I have already sent the link to the page containing the list of ideas > >> for > >> > implementation of new API that I propose for this project. Would you > >> like > >> > to add something to it or give some comments on it? The Gsoc proposal > >> > deadline is 9th of April and I am planning to put all of this in my > >> > proposal to make it effective . So any more ideas or comments will be > >> > really helpful for me . Here is the link to the page: > >> > https://github.com/ananyashreyjain/Gsoc-Scipy/wiki/Gsoc-2019:-Scipy . > >> > > >> > On Sun, Mar 31, 2019 at 5:23 AM <scipy-dev-request(a)python.org> wrote: > >> > > >> >> > >> >> Message: 1 > >> >> Date: Sat, 30 Mar 2019 13:31:51 -0400 > >> >> From: Eric Larson <larson.eric.d(a)gmail.com> > >> >> To: SciPy Developers List <scipy-dev(a)python.org> > >> >> Subject: Re: [SciPy-Dev] GSoC 2019 > >> >> Message-ID: > >> >> < > >> >> CAGu2niVWsKrMukfWcAWAtj-1NBm2utCeh3BOnNku7jNoAcx-kw(a)mail.gmail.com> > >> >> Content-Type: text/plain; charset="utf-8" > >> >> > >> >> It looks like the main ideas are in these bits of code, but not > >> explicitly > >> >> stated. In brief it looks like the proposed API is to add a > >> `fft_object` > >> >> kwarg to every function in `scipy.fftpack`. It would be good to > discuss > >> >> this choice explicitly -- what advantages and disadvantages it has > >> >> relative > >> >> to other approaches (e.g., creating new namespaces, or having a > >> >> `set_backend` function (and/or context manager) that would change > >> things > >> >> under the hood). > >> >> > >> >> If you make changes and need more feedback, let us know. The more > >> specific > >> >> you can be about what you need feedback on (high level ideas, low > level > >> >> implementation, etc.), the better. > >> >> > >> >> Eric > >> >> > >> >> > >> >> On Fri, Mar 29, 2019 at 4:12 AM Anany Shrey Jain < > >> >> f20170920(a)goa.bits-pilani.ac.in> wrote: > >> >> > >> >> > Hey, > >> >> > I went through the idea list of Scipy for this years's GSoC and I > >> intend > >> >> > to work towards the idea of revamping the Scipy's fftpack. I have > >> >> already > >> >> > been contributing to Scipy since the end of last year and I have > some > >> >> > knowledge about Fourier transforms as it was the part of my > >> discipline. > >> >> I > >> >> > could not take part in this discussion earlier because my mid > >> semester > >> >> > exams were going on, but since they are now over so I can devote as > >> much > >> >> > time as required on this project. I have already created a github > >> page > >> >> with > >> >> > some code snippets and functionalities that we can implement, here > is > >> >> the > >> >> > link to it: > >> >> > > https://github.com/ananyashreyjain/Gsoc-Scipy/wiki/Gsoc-2019:-Scipy > >> . > >> >> > > >> >> > Cheers, > >> >> > Anany Jain > >> >> > > >> >> > _______________________________________________ > >> >> > SciPy-Dev mailing list > >> >> > SciPy-Dev(a)python.org > >> >> > https://mail.python.org/mailman/listinfo/scipy-dev > >> >> > > >> >>

4 3

Fix ndimage.rotate plane check
by Riadh 09 Apr '19

09 Apr '19

Hello, I opened the issue #10011 <https://github.com/scipy/scipy/issues/10011> about wrong validation of rotation plane axes in the ndimage.interpolation.rotate function. I also made some propositions for refactoring this function. You can see my contribution in the pull request #10028 <https://github.com/scipy/scipy/pull/10028>. This is my first time filling a pull request for Scipy, so I hope that every thing is OK with it. Thank you for your review and comments, Riadh.

2 1

GSoC 2019 Projects
by Peter Bell 07 Apr '19

07 Apr '19

Hello, I am interested in applying for Google summer of code and was wondering if you are still looking for students? A bit about myself ------------------- My name is Peter and I'm in my 4th (penultimate) year of an integrated masters in computational physics. Programming and numerical methods forms a key part of this degree and many of my courses make heavy use of python with cython, numpy, scipy, matplotlib, etc. While my degree is primarily Physics, I have taken a number of computer science courses covering software engineering, object oriented design and high performance programming. I also have an interest in open source collaboration and, when time has allowed, have contributed to a few projects via github (@peterbell10). The majority of my contributions have been to a minecraft server which I've been contributing to since summer 2017. You can see my pull requests here: https://github.com/cuberite/cuberite/pulls?q=is%3Apr+is%3Amerged+author%3Ap… Questions about the projects ----------------------------- I have some questions regarding both of the proposed projects. Revamp fftpack: * Is the primary intention to keep the scipy.fftpack interface the same, but add some mechanism to inject custom implementations? Or will this require changes to the front-facing interface as well? (e.g. for agreement with numpy.fft) * What should this look like from the user's point of view? e.g. import scipy.fftpack import pyFFTW.interfaces.scipy_fftpack as fftw_pack scipy.fftpack.set_backend(fftw_pack) where we just provide a module implementing a well-documented backend interface. Alternatively, fftw_pack could be an object, potentially with its own instances of caches and settings which might allow for more configuration from the user. Improve ODE solvers (mainly the first 3 tasks): * How much mathematical background is required? I have done a course covering the mathematics of ODE schemes including RK methods and some variable timestep methods, but am not familiar with control theory which is mentioned for the 3rd task. * Will developing the test set just involve binding existing implementations to python for use in a testing framework, or do you expect these to be rewritten entirely in python or cython? Best regards, Peter

4 6

GSoC 2019
by Anany Shrey Jain 05 Apr '19

05 Apr '19

Hey Greg, The project "Revamp FFTPACK" also aims at contributing to third party libraries to fill in functionality gaps. For pyFFTW I can see that work on adding Real-to-Real transforms has been going on for a long time. So do I need to continue the work going on in https://github.com/pyFFTW/pyFFTW/pull/256 as a part of this project? What needs to be done for the other two libraries? Do I need to make a similar PR to other two libraries as well? For all the three libraries there are SciPy-compatible routines already defined. After getting required information using the newly created interface I need to use them at the backend, right? -Anany > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 4 Apr 2019 10:58:27 -0400 > From: Gregory Lee <grlee77(a)gmail.com> > To: SciPy Developers List <scipy-dev(a)python.org> > Subject: Re: [SciPy-Dev] SciPy-Dev Digest, Vol 185, Issue 18 > Message-ID: > < > CAJR3sXcJuqQJrDboN5zH-pd7NG-wD8EWCdTCAtQ+vJpnZjKsRw(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi Anany, > > It is nice to see that you have updated with multiple potential APIs. Can > some other core SciPy devs also help give feedback on what SciPy would find > preferable? Out of the four you proposed, I think something along the lines > of the second method is cleanest in terms of setting the backend at > runtime. I think having some aspect from your 4th variant where the default > backend can be read from a configuration file and/or environment variable > is also desirable so that the user's desired default can always be > automatically set at import time. > > I disagree with the following specific statement, though: > "It is better to use all the above mentioned ideas because every idea has > its own benefits besides it gives user the luxury to choose the method > implementation of backend." > > I do not think we want to provide multiple ways to do the same thing. It > can be confusing to users and more work for devs to maintain. I think there > should be one method of setting the backend that the user can either call > at run time or have it be automatically called based on some sort of > configuration file and/or environment variable(s) at import time. > > - Greg > > > On Tue, Apr 2, 2019 at 2:19 PM Anany Shrey Jain < > f20170920(a)goa.bits-pilani.ac.in> wrote: > > > Hey, > > I have already sent the link to the page containing the list of ideas for > > implementation of new API that I propose for this project. Would you like > > to add something to it or give some comments on it? The Gsoc proposal > > deadline is 9th of April and I am planning to put all of this in my > > proposal to make it effective . So any more ideas or comments will be > > really helpful for me . Here is the link to the page: > > https://github.com/ananyashreyjain/Gsoc-Scipy/wiki/Gsoc-2019:-Scipy . > > > > On Sun, Mar 31, 2019 at 5:23 AM <scipy-dev-request(a)python.org> wrote: > > > >> > >> Message: 1 > >> Date: Sat, 30 Mar 2019 13:31:51 -0400 > >> From: Eric Larson <larson.eric.d(a)gmail.com> > >> To: SciPy Developers List <scipy-dev(a)python.org> > >> Subject: Re: [SciPy-Dev] GSoC 2019 > >> Message-ID: > >> < > >> CAGu2niVWsKrMukfWcAWAtj-1NBm2utCeh3BOnNku7jNoAcx-kw(a)mail.gmail.com> > >> Content-Type: text/plain; charset="utf-8" > >> > >> It looks like the main ideas are in these bits of code, but not > explicitly > >> stated. In brief it looks like the proposed API is to add a `fft_object` > >> kwarg to every function in `scipy.fftpack`. It would be good to discuss > >> this choice explicitly -- what advantages and disadvantages it has > >> relative > >> to other approaches (e.g., creating new namespaces, or having a > >> `set_backend` function (and/or context manager) that would change things > >> under the hood). > >> > >> If you make changes and need more feedback, let us know. The more > specific > >> you can be about what you need feedback on (high level ideas, low level > >> implementation, etc.), the better. > >> > >> Eric > >> > >> > >> On Fri, Mar 29, 2019 at 4:12 AM Anany Shrey Jain < > >> f20170920(a)goa.bits-pilani.ac.in> wrote: > >> > >> > Hey, > >> > I went through the idea list of Scipy for this years's GSoC and I > intend > >> > to work towards the idea of revamping the Scipy's fftpack. I have > >> already > >> > been contributing to Scipy since the end of last year and I have some > >> > knowledge about Fourier transforms as it was the part of my > discipline. > >> I > >> > could not take part in this discussion earlier because my mid semester > >> > exams were going on, but since they are now over so I can devote as > much > >> > time as required on this project. I have already created a github page > >> with > >> > some code snippets and functionalities that we can implement, here is > >> the > >> > link to it: > >> > https://github.com/ananyashreyjain/Gsoc-Scipy/wiki/Gsoc-2019:-Scipy . > >> > > >> > Cheers, > >> > Anany Jain > >> > > >> > _______________________________________________ > >> > SciPy-Dev mailing list > >> > SciPy-Dev(a)python.org > >> > https://mail.python.org/mailman/listinfo/scipy-dev > >> > > >>

2 1

NetworkX 2.3rc3 released (Python 3 only)
by Jarrod Millman 05 Apr '19

05 Apr '19

I am happy to announce the third **release candidate** for NetworkX 2.3! NetworkX is a Python package for the creation, manipulation, and study of the structure, dynamics, and functions of complex networks. This release supports Python 3.5-3.7 (i.e., this is our first **Python 3 only** release). Please try out the pre-release and let us know about any problems you find. If no major issues arise, we will release 2.3 final next week. Please see the draft of the 2.3 release announcement: https://networkx.github.io/documentation/latest/release/release_dev.html Since this is a pre-release, pip won't automatically install it. So $ pip install networkx still installs networkx-2.2. But $ pip install --pre networkx will install networkx-2.3rc3. If you already have networkx installed then you need to do $ pip install --pre --upgrade networkx For more information, please visit our `website <http://networkx.github.io/>`_ and our `gallery of examples <https://networkx.github.io/documentation/latest/auto_examples/index.html>`_. Please send comments and questions to the `networkx-discuss mailing list <http://groups.google.com/group/networkx-discuss>`_. Best regards, Jarrod

1 0

Re: [SciPy-Dev] SciPy-Dev Digest, Vol 185, Issue 18
by Anany Shrey Jain 04 Apr '19

04 Apr '19

Hey, I have already sent the link to the page containing the list of ideas for implementation of new API that I propose for this project. Would you like to add something to it or give some comments on it? The Gsoc proposal deadline is 9th of April and I am planning to put all of this in my proposal to make it effective . So any more ideas or comments will be really helpful for me . Here is the link to the page: https://github.com/ananyashreyjain/Gsoc-Scipy/wiki/Gsoc-2019:-Scipy . On Sun, Mar 31, 2019 at 5:23 AM <scipy-dev-request(a)python.org> wrote: > > Message: 1 > Date: Sat, 30 Mar 2019 13:31:51 -0400 > From: Eric Larson <larson.eric.d(a)gmail.com> > To: SciPy Developers List <scipy-dev(a)python.org> > Subject: Re: [SciPy-Dev] GSoC 2019 > Message-ID: > < > CAGu2niVWsKrMukfWcAWAtj-1NBm2utCeh3BOnNku7jNoAcx-kw(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > It looks like the main ideas are in these bits of code, but not explicitly > stated. In brief it looks like the proposed API is to add a `fft_object` > kwarg to every function in `scipy.fftpack`. It would be good to discuss > this choice explicitly -- what advantages and disadvantages it has relative > to other approaches (e.g., creating new namespaces, or having a > `set_backend` function (and/or context manager) that would change things > under the hood). > > If you make changes and need more feedback, let us know. The more specific > you can be about what you need feedback on (high level ideas, low level > implementation, etc.), the better. > > Eric > > > On Fri, Mar 29, 2019 at 4:12 AM Anany Shrey Jain < > f20170920(a)goa.bits-pilani.ac.in> wrote: > > > Hey, > > I went through the idea list of Scipy for this years's GSoC and I intend > > to work towards the idea of revamping the Scipy's fftpack. I have already > > been contributing to Scipy since the end of last year and I have some > > knowledge about Fourier transforms as it was the part of my discipline. I > > could not take part in this discussion earlier because my mid semester > > exams were going on, but since they are now over so I can devote as much > > time as required on this project. I have already created a github page > with > > some code snippets and functionalities that we can implement, here is the > > link to it: > > https://github.com/ananyashreyjain/Gsoc-Scipy/wiki/Gsoc-2019:-Scipy . > > > > Cheers, > > Anany Jain > > > > _______________________________________________ > > SciPy-Dev mailing list > > SciPy-Dev(a)python.org > > https://mail.python.org/mailman/listinfo/scipy-dev > > >

2 1

Re: [SciPy-Dev] Google Season of Docs
by Christoph Baumgarten 04 Apr '19

04 Apr '19

Sure, I will add it in the next few days. On Tue, Apr 2, 2019 at 6:02 PM <scipy-dev-request(a)python.org> wrote: > Send SciPy-Dev mailing list submissions to > scipy-dev(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scipy-dev > or, via email, send a message with subject or body 'help' to > scipy-dev-request(a)python.org > > You can reach the person managing the list at > scipy-dev-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of SciPy-Dev digest..." > > > Today's Topics: > > 1. Re: Google Season of Docs (Ralf Gommers) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 2 Apr 2019 10:40:20 +0200 > From: Ralf Gommers <ralf.gommers(a)gmail.com> > To: SciPy Developers List <scipy-dev(a)python.org> > Subject: Re: [SciPy-Dev] Google Season of Docs > Message-ID: > <CABL7CQipvttnWrAE= > V6Fr+iJKjGM1x0wgKbK-ea7ZNsOXx6YDg(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > On Fri, Mar 22, 2019 at 8:07 PM Ralf Gommers <ralf.gommers(a)gmail.com> > wrote: > > > > > > > On Thu, Mar 21, 2019 at 12:10 PM Christoph Baumgarten < > > christoph.baumgarten(a)gmail.com> wrote: > > > >> > >> Hi, > >> > >> taking up Ralf's mail on GSoD, I had a look at the website and propose > to > >> apply. > >> > >> My project suggestion would be to work on the documentation of a module > >> like stats: > >> > > > >> - improve the tutorial site > >> https://docs.scipy.org/doc/scipy/reference/tutorial/stats.html > >> - improve the documentation of the API > >> - check whether current descriptions can be improved > >> - add examples and references > >> - cross-ref to other relevant / similar functions in the module > >> > >> The second point would require decent knowledge of statistics whereas > the > >> first point should be easier to tackle for a technical writer. So I see > the > >> second point as optional and one could also try to work on the tutorial > >> site of more than one module during the 3-month period. > >> > > > >> The task would leave quite a bit of freedom to the writer to choose what > >> areas to focus on, potentially taking up some of the documentation > issues > >> that are already on Github. > >> > >> I could assist as a mentor for stats (we need at least two mentors). > >> > >> Any suggestions or comments? > >> > > > > Thanks Christoph, this sounds like a good idea to me. We have a lot of > > content on scipy.stats, and a tech writer could really make a difference > in > > making that easier to understand for users. I suggest we write this up as > > an idea on our (to be created) ideas page as soon as we know whether we > > have an umbrella organization that we can participate with. > > > > An update on this: NumFOCUS is applying as an umbrella org. We'll need to > add our ideas at > https://github.com/numfocus/gsod/blob/master/2019/SciPy_ideas_list.md. > Since there are going to be few spots available, it's the quality of the > content that matters for acceptance of NumFOCUS as an umbrella org. We > should have 1-2 ideas that are well described up on that page by April > 14th. There are some guidelines at > > https://developers.google.com/season-of-docs/docs/project-ideas#project-idea > > Christoph, could you add your idea in that SciPy_ideas_list.md file? > > I will add some more general information there about getting started on the > SciPy docs. Also, if anyone has another idea, that would be great to hear. > > Cheers, > Ralf >

3 2