[SciPy-Dev] SciPy-Dev Digest, Vol 185, Issue 18

[Anany Shrey Jain]

Hey,
I have already sent the link to the page containing the list of ideas for
implementation of new API that I propose for this project. Would you like
to add something to it or give some comments on it? The Gsoc proposal
deadline is 9th of April and I am planning to put all of this in my
proposal to make it effective . So any more ideas or comments will be
really helpful for me . Here is the link to the page:
https://github.com/ananyashreyjain/Gsoc-Scipy/wiki/Gsoc-2019:-Scipy .

On Sun, Mar 31, 2019 at 5:23 AM <scipy-dev-request at python.org> wrote:

>
> Message: 1
> Date: Sat, 30 Mar 2019 13:31:51 -0400
> From: Eric Larson <larson.eric.d at gmail.com>
> To: SciPy Developers List <scipy-dev at python.org>
> Subject: Re: [SciPy-Dev] GSoC 2019
> Message-ID:
>         <
> CAGu2niVWsKrMukfWcAWAtj-1NBm2utCeh3BOnNku7jNoAcx-kw at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> It looks like the main ideas are in these bits of code, but not explicitly
> stated. In brief it looks like the proposed API is to add a `fft_object`
> kwarg to every function in `scipy.fftpack`. It would be good to discuss
> this choice explicitly -- what advantages and disadvantages it has relative
> to other approaches (e.g., creating new namespaces, or having a
> `set_backend` function (and/or context manager) that would change things
> under the hood).
>
> If you make changes and need more feedback, let us know. The more specific
> you can be about what you need feedback on (high level ideas, low level
> implementation, etc.), the better.
>
> Eric
>
>
> On Fri, Mar 29, 2019 at 4:12 AM Anany Shrey Jain <
> f20170920 at goa.bits-pilani.ac.in> wrote:
>
> > Hey,
> > I went through the idea list of Scipy for this years's GSoC and I intend
> > to work towards the idea of revamping the Scipy's fftpack. I have already
> > been contributing to Scipy since the end of last year and I have some
> > knowledge about Fourier transforms as it was the part of my discipline. I
> > could not take part in this discussion earlier because my mid semester
> > exams were going on, but since they are now over so I can devote as much
> > time as required on this project. I have already created a github page
> with
> > some code snippets and functionalities that we can implement, here is the
> > link to it:
> > https://github.com/ananyashreyjain/Gsoc-Scipy/wiki/Gsoc-2019:-Scipy .
> >
> > Cheers,
> > Anany Jain
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://mail.python.org/pipermail/scipy-dev/attachments/20190330/f1755dfb/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 2
> Date: Sat, 30 Mar 2019 19:38:41 -0400
> From: Gregory Lee <grlee77 at gmail.com>
> To: SciPy Developers List <scipy-dev at python.org>
> Subject: Re: [SciPy-Dev] GSoC 2019
> Message-ID:
>         <CAJR3sXfnnr-0JO87gLbhh-7b-w=KjaNpac+F+FHTeFt2K=_
> QMw at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> On Fri, Mar 29, 2019 at 4:12 AM Anany Shrey Jain <
> f20170920 at goa.bits-pilani.ac.in> wrote:
>
> > Hey,
> > I went through the idea list of Scipy for this years's GSoC and I intend
> > to work towards the idea of revamping the Scipy's fftpack. I have already
> > been contributing to Scipy since the end of last year and I have some
> > knowledge about Fourier transforms as it was the part of my discipline. I
> > could not take part in this discussion earlier because my mid semester
> > exams were going on, but since they are now over so I can devote as much
> > time as required on this project. I have already created a github page
> with
> > some code snippets and functionalities that we can implement, here is the
> > link to it:
> > https://github.com/ananyashreyjain/Gsoc-Scipy/wiki/Gsoc-2019:-Scipy .
> >
> > Cheers,
> > Anany Jain
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
>
>
> Hi Anany,
>
> Thank you for your interest in GSOC with SciPy!  It's great to see that you
> are already thinking about a potential API and learning about the 3rd party
> FFT libraries. Along the lines of Eric's comment, think about what this API
> choice implies for users of libraries built upon SciPy. Would users of 3rd
> party libraries that rely on SciPy's FFTs be immediately able to take
> advantage of a differentf FFT backend in this scenario? (in other words,
> the case where the user is not calling scipy.fftpack.fft directly but is
> using it only indirectly via a function in a downstream library).
>
> Matplotlib is an example of an existing Python library supporting multiple
> backends that can be considered for comparison (see:
> https://matplotlib.org/faq/usage_faq.html#what-is-a-backend). Think about
> the relative benefits/drawbacks of some of the methods they use for setting
> a backend there as compared to the approach proposed for FFTs here.
>
> Another thing that should be considered: pyFFTW, MKL and CuPy already have
> some functions that match the existing scipy.fftpack API, but they may have
> one or more additional arguments particular to features of that library.
> How would you propose to deal with supporting additional optional arguments
> on the SciPy end?
>
> One other minor comment: I know that the blog post you made may be serving
> partly as personal notes, but please do not directly copy sections of text
> (e.g. the description of BLAS/LAPACK libraries) from elsewhere without
> attribution when writing a final proposal. (Also, the FFT functions are
> separate from the BLAS/LAPACK functions in MKL, so you will not need to
> discuss BLAS or LAPACK specifically in your proposal)
>
> - Greg
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://mail.python.org/pipermail/scipy-dev/attachments/20190330/c1351e7e/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 3
> Date: Sat, 30 Mar 2019 19:53:02 -0400
> From: Gregory Lee <grlee77 at gmail.com>
> To: SciPy Developers List <scipy-dev at python.org>
> Subject: Re: [SciPy-Dev] GSoC 2019
> Message-ID:
>         <CAJR3sXdcPMAn_DETF8W-=
> MWnGbkJMpQ2hK1ni20zoLbgf6S0Yw at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi Gopi,
>
> I am not sure I follow exactly what you are suggesting here. Are you
> proposing that the `fftpackN` variable in your example would be a Python
> module corresponding to a given backend? If so, why is an array of data, z,
> being passed into the newAPI function here? I don't see why a data array
> should be needed at all to choose a backend.
>
> Please also see my response to Anany's recent post on this list for a
> couple of other suggestions of things to consider in terms of the API.
>
> - Greg
>
>
> On Fri, Mar 29, 2019 at 5:27 AM Gopi Manohar Tatiraju <
> deathcoderx at gmail.com>
> wrote:
>
> > Hey,
> > I went through the code and got some ideas.
> > So what will happen is:
> > First, we will declare FFTW as we use them normally.
> >
> > z = np.random.rand(2, 128, 64)
> > pyfftw.FFTW(z, axes=(-1, ))
> > ..
> > ..
> > Now after declaring our basic work to shift to another FFTpack, what we
> > can do is:
> > fftPackN= newAPI(z, pack = 'PYFFTW');
> >
> > Pack will take the name of the 3rd party FFT pack which we want to use.
> > Suggestions needed.
> >
> > Cheers.
> >
> >
>
> >
> > On Wed, Mar 27, 2019 at 8:43 PM Eric Larson <larson.eric.d at gmail.com>
> > wrote:
> >
> >> After reading the related issues and thinking about the problem, perhaps
> >> the best first step is you come up with some initial API proposal.
> >>
> >> One way to work through the process is to think about the use cases that
> >> we want to support. How should users actually write code to use the new
> >> functionality? You can start with short code snippets that do not work
> >> currently, but should work for users at the end of the project. Then
> think
> >> about what needs to change inside SciPy for these code snippets to work.
> >> This can become an iterative process as you realize that the SciPy
> changes
> >> might be better or easier with changes to the user-facing API, etc.
> >>
> >> Eric
> >>
> >>
> >> On Tue, Mar 26, 2019 at 3:38 PM Gopi Manohar Tatiraju <
> >> deathcoderx at gmail.com> wrote:
> >>
> >>> Hey,
> >>> As proposal submission period started I want to discuss a bit anobt how
> >>> can we implement the API.
> >>> I would very much appriciate inputs and guidance.
> >>>
> >>> Cheers.
> >>>
> >>> On Thu, Mar 7, 2019 at 11:40 PM Eric Larson <larson.eric.d at gmail.com>
> >>> wrote:
> >>>
> >>>> The best way to figure that out is probably to take a look at some of
> >>>> the existing (minor) SciPy GitHub issues and try to work on one or
> two that
> >>>> you feel like working on. Actually trying to work out how to
> contribute
> >>>> code changes (as independently as possible/reasonable based on the
> existing
> >>>> guides/docs already discoverable online) will quickly highlight
> things that
> >>>> need to be learned. Some of the general things are the SciPy
> contributing
> >>>> guidelines, standard GitHub PR workflow, communication with
> reviewers, and
> >>>> so forth.
> >>>>
> >>>> For this particular project, it's important to understand the various
> >>>> flavors and properties of FFT in SciPy (real, complex, n-dimensional,
> >>>> caching plans, etc.) as well as other projects like NumPy, CuPy, and
> >>>> pyFFTW. Some of this can be learned and improved upon as part of
> GSoC, but
> >>>> the more you know and can demonstrate / put into practice knowledge of
> >>>> before the applications are reviewed, the better.
> >>>>
> >>>> Eric
> >>>>
> >>>>
> >>>> On Wed, Mar 6, 2019 at 4:55 AM Gopi Manohar Tatiraju <
> >>>> deathcoderx at gmail.com> wrote:
> >>>>
> >>>>> Thank you for the detailed reply.
> >>>>> I will keep these point in mind while preparing my proposal. Apart
> >>>>> from proposal, what skills shall I improve so that I can contribute
> more
> >>>>> effectively and efficiently.
> >>>>>
> >>>>> Cheers.
> >>>>>
> >>>>>
> >>>>> On Wed, Mar 6, 2019, 3:42 AM Eric Larson <larson.eric.d at gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>>> There are a number of considerations that would be good to address.
> >>>>>> I'd start by considering the things that are typically considered
> >>>>>> when making changes to SciPy
> >>>>>> <
> https://docs.scipy.org/doc/scipy/reference/hacking.html#contributing-to-scipy
> >,
> >>>>>> but I'd expand it to a longer list since the scope is bigger than a
> typical
> >>>>>> single PR. The list would contain (at least):
> >>>>>>
> >>>>>> - what will the user-facing API / contract be?
> >>>>>> - what in SciPy needs to change to allow it (internals /
> >>>>>> implementation outline, perhaps skeleton functions/classes)?
> >>>>>> - what examples should we show users (in general this ends up being
> >>>>>> just simple use cases of the API)?
> >>>>>> - how will we test it for correctness?
> >>>>>> - how will we benchmark it to prevent performance regressions?
> >>>>>> - how can we break up the work for these steps (if possible) into
> >>>>>> separable, sequential (or -- better but not usually possible --
> >>>>>> independent) PRs?
> >>>>>>
> >>>>>> You might notice that none of these points are actually all that
> >>>>>> specific to the FFT project, but can be generally applicable to GSoC
> >>>>>> projects. For the FFT project, I would say "how do we build the
> backend" is
> >>>>>> indeed implicitly contained / outlined in these bullet points. And
> as with
> >>>>>> most projects, the first two points (and possibly the last) will
> probably
> >>>>>> be the most challenging to sort out.
> >>>>>>
> >>>>>> Eric
> >>>>>>
> >>>>>>
> >>>>>> On Mon, Mar 4, 2019 at 10:09 PM Gopi Manohar Tatiraju <
> >>>>>> deathcoderx at gmail.com> wrote:
> >>>>>>
> >>>>>>> Hey Eric,
> >>>>>>> So for my proposal, shall I include how can we built a back end?
> >>>>>>> What all points shall I include to make it effective?
> >>>>>>>
> >>>>>>> Cheers.
> >>>>>>>
> >>>>>>> On Tue, Mar 5, 2019, 4:50 AM Eric Larson <larson.eric.d at gmail.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> That would be me -- don't worry, I'm on this list, as are
> >>>>>>>> (presumably) other developers interested in the discussion, so no
> need to
> >>>>>>>> document elsewhere for now from our end. It looks like Greg has
> summarized
> >>>>>>>> the current status and thoughts we've had, so I don't have much
> to add at
> >>>>>>>> this point.
> >>>>>>>>
> >>>>>>>> Eric
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Sun, Mar 3, 2019 at 11:59 PM Gopi Manohar Tatiraju <
> >>>>>>>> deathcoderx at gmail.com> wrote:
> >>>>>>>>
> >>>>>>>>> Hey Greg,
> >>>>>>>>> I got the basic idea about what we want to implement this year.
> >>>>>>>>> Our first priority is to create a backend for 3rd party fftpacks.
> >>>>>>>>> We can create API using Python and Flask or any other suitable
> framework.
> >>>>>>>>>
> >>>>>>>>> I thought this discussion was going on on SciPy Developer List
> >>>>>>>>> only. So do I need to document it all and post it somewhere so
> that other
> >>>>>>>>> contributors can see this?
> >>>>>>>>> Also, I couldn't find any contact info regarding Eric. It would
> be
> >>>>>>>>> very helpful if we include him in this discussion as this is a
> roadmap for
> >>>>>>>>> GSoC project and I can also get more insight into the project
> and expected
> >>>>>>>>> outcomes.
> >>>>>>>>>
> >>>>>>>>> Cheers.
> >>>>>>>>>
> >>>>>>>>> On Mon, Mar 4, 2019 at 9:31 AM Gregory Lee <grlee77 at gmail.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Sun, Mar 3, 2019 at 12:28 PM Gopi Manohar Tatiraju <
> >>>>>>>>>> deathcoderx at gmail.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hey Gregory,
> >>>>>>>>>>> I went through the idea and I am very much interested in
> working
> >>>>>>>>>>> on it. I had Fourier Transforms as a part of my academic
> course, so I am
> >>>>>>>>>>> well aware with basic concepts.
> >>>>>>>>>>> One thing I want to discuss is, its mentioned that we have to
> >>>>>>>>>>> create a backend system for 3rd party FFTpacks so my dobuts
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> 1. How are we going to create API?
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> This will needed to be decided on in consensus with the SciPy
> >>>>>>>>>> developers and user community. I am mentoring for the project
> as a
> >>>>>>>>>> contributor here and a maintainer of the related pyFFTW
> package, but I am
> >>>>>>>>>> not a core SciPy dev (the other mentor, Eric, is a core dev
> here). I have
> >>>>>>>>>> not discussed the API in any detail with the SciPy team at this
> stage.
> >>>>>>>>>> Working out details of what the API should look like and
> working to
> >>>>>>>>>> finalize that in collaboration with the community will be a
> primary task
> >>>>>>>>>> during the early part of the GSOC project.
> >>>>>>>>>>
> >>>>>>>>>> 2. After the creation of backend will user at the time of
> >>>>>>>>>>> installation of SciPy will decide what all 3rd party FFTpacks
> he needs or
> >>>>>>>>>>> we will include some by default?
> >>>>>>>>>>>
> >>>>>>>>>>> The idea is to allow users to opt-in to a different FFT backend
> >>>>>>>>>> at run time, but these alternative libraries will not be
> distributed with
> >>>>>>>>>> SciPy itself due to incompatible software licenses (pyFFTW and
> mkl-fft) or
> >>>>>>>>>> a requirement for specific hardware (CuPy requires an NVIDIA
> GPU). SciPy
> >>>>>>>>>> will continue to offer its own bundled FFT library (currently
> fftpack, but
> >>>>>>>>>> this could be switched to pocketfft as discussed in the ideas
> page). The
> >>>>>>>>>> purpose of the backend interface is to provide a convenient way
> for users
> >>>>>>>>>> to have third party libraries perform the FFTs instead, but
> under the
> >>>>>>>>>> existing scipy.fftpack API. This is motivated by a desire by
> end users and
> >>>>>>>>>> downstream projects to have features such as multi-threading
> (pyFFTW and
> >>>>>>>>>> mkl-fft) or GPU support (CuPy) that are not implemented in
> scipy.fftpack.
> >>>>>>>>>> Ultimately it will be up to users that opt-in to using these
> other
> >>>>>>>>>> libraries to confirm that the license terms and are compatible
> with their
> >>>>>>>>>> use case.
> >>>>>>>>>>
> >>>>>>>>>> p.s. It is not a big deal, but I think the SciPy-Dev mailing
> >>>>>>>>>> lists prefers replies **below** the email you are replying to
> so people can
> >>>>>>>>>> more easily follow the thread of discussion. There are some
> guidelines at:
> >>>>>>>>>> https://www.scipy.org/scipylib/mailing-lists.html
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> I will research more about how to implement a backend myself and
> >>>>>>>>>>> will let you know as soon as I come up with something new.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> CHeers,
> >>>>>>>>>>> Greg
> >>>>>>>>>>>
> >>>>>>>>>>> On Sun, Mar 3, 2019 at 10:25 PM Gregory Lee <grlee77 at gmail.com
> >
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> On Sun, Mar 3, 2019 at 4:19 AM Gopi Manohar Tatiraju <
> >>>>>>>>>>>> deathcoderx at gmail.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> I want to work on SciPy in GSoC 2019. I started fixing some
> >>>>>>>>>>>>> bugs, also made some pull request. Can anyone suggest me how
> should I
> >>>>>>>>>>>>> proceed further?
> >>>>>>>>>>>>> _______________________________________________
> >>>>>>>>>>>>> SciPy-Dev mailing list
> >>>>>>>>>>>>> SciPy-Dev at python.org
> >>>>>>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Hi Gopi,
> >>>>>>>>>>>>
> >>>>>>>>>>>> I think you have probably already found it, but the GSOC
> >>>>>>>>>>>> project we have already identified mentors for in SciPy is
> here:
> >>>>>>>>>>>> https://github.com/scipy/scipy/wiki/GSoC-2019-project-ideas
> >>>>>>>>>>>>
> >>>>>>>>>>>> If this is a project you are interested in then I would start
> >>>>>>>>>>>> thinking about what the FFT interface would look like and
> reviewing some of
> >>>>>>>>>>>> the past/present issues related to fftpack / FFT in SciPy.
> Please ask
> >>>>>>>>>>>> questions you have related to that project here so that you
> can clarify
> >>>>>>>>>>>> what would be involved in writing a successful application.
> >>>>>>>>>>>>
> >>>>>>>>>>>> If you have a different idea in mind you can propose it here,
> >>>>>>>>>>>> but you should do that soon, because it will be necessary to
> determine if
> >>>>>>>>>>>> there is interest from the SciPy team before you spend time
> coming up with
> >>>>>>>>>>>> a detailed proposal. We have to determine if a project sounds
> feasible to
> >>>>>>>>>>>> complete for GSOC and if appropriate mentors can be
> identified.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Ultimately, you will need to submit an application to SciPy
> >>>>>>>>>>>> between March 25th - April 9th.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Cheers,
> >>>>>>>>>>>> Greg
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> _______________________________________________
> >>>>>>>>>>>> SciPy-Dev mailing list
> >>>>>>>>>>>> SciPy-Dev at python.org
> >>>>>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
> >>>>>>>>>>>>
> >>>>>>>>>>> _______________________________________________
> >>>>>>>>>>> SciPy-Dev mailing list
> >>>>>>>>>>> SciPy-Dev at python.org
> >>>>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
> >>>>>>>>>>>
> >>>>>>>>>> _______________________________________________
> >>>>>>>>>> SciPy-Dev mailing list
> >>>>>>>>>> SciPy-Dev at python.org
> >>>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
> >>>>>>>>>>
> >>>>>>>>> _______________________________________________
> >>>>>>>>> SciPy-Dev mailing list
> >>>>>>>>> SciPy-Dev at python.org
> >>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
> >>>>>>>>>
> >>>>>>>> _______________________________________________
> >>>>>>>> SciPy-Dev mailing list
> >>>>>>>> SciPy-Dev at python.org
> >>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
> >>>>>>>>
> >>>>>>> _______________________________________________
> >>>>>>> SciPy-Dev mailing list
> >>>>>>> SciPy-Dev at python.org
> >>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
> >>>>>>>
> >>>>>> _______________________________________________
> >>>>>> SciPy-Dev mailing list
> >>>>>> SciPy-Dev at python.org
> >>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
> >>>>>>
> >>>>> _______________________________________________
> >>>>> SciPy-Dev mailing list
> >>>>> SciPy-Dev at python.org
> >>>>> https://mail.python.org/mailman/listinfo/scipy-dev
> >>>>>
> >>>> _______________________________________________
> >>>> SciPy-Dev mailing list
> >>>> SciPy-Dev at python.org
> >>>> https://mail.python.org/mailman/listinfo/scipy-dev
> >>>>
> >>> _______________________________________________
> >>> SciPy-Dev mailing list
> >>> SciPy-Dev at python.org
> >>> https://mail.python.org/mailman/listinfo/scipy-dev
> >>>
> >> _______________________________________________
> >> SciPy-Dev mailing list
> >> SciPy-Dev at python.org
> >> https://mail.python.org/mailman/listinfo/scipy-dev
> >>
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://mail.python.org/pipermail/scipy-dev/attachments/20190330/54f38250/attachment.html
> >
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
> ------------------------------
>
> End of SciPy-Dev Digest, Vol 185, Issue 18
> ******************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20190402/e1f35f09/attachment-0001.html>