[SciPy-Dev] GSoC 2019

Tue Apr 2 02:47:11 EDT 2019

On Tue, Apr 2, 2019 at 8:17 AM Gopi Manohar Tatiraju <deathcoderx at gmail.com>
wrote:

> Hey,
> Greg, I went through the ideas you have mentioned and also looked up the
> matplotlib docs as suggested.  And I feel like creating a new namespace
> would be a better option as it won't require many changes in the current
> implementation. As matplotlib.pyplot is a well-implemented module  I feel
> like we can consider it as our basic structure and start implementing the
> backend.
>

A new namespace is not a good idea. There's no reason that things can't
work for the current namespace.

> For the start, I think import can look like this(rough idea)
> import scipy.fftpack.pyfftw as fft
> where pyffw can be replaced by any third party fftpack.
>

We don't want a PyFFTW namespace or direct support for PyFFTW (note that we
removed such support years ago due to license reasons). Instead, we want to
allow third parties to implement a backend; for FFTW this will live in
https://github.com/pyFFTW/pyFFTW

Cheers,
Ralf

> Cheers.
> -Gopi
>
>
> On Sun, Mar 31, 2019 at 2:00 AM Gopi Manohar Tatiraju <
> deathcoderx at gmail.com> wrote:
>
>> Hey,
>> I feel like I need to make some more changes regarding the idea. I went
>> through your reply to Anany and I will consider all those points and will
>> get back to you once I come up with something.
>>
>> -Gopi.
>>
>> On Sun, Mar 31, 2019 at 5:24 AM Gregory Lee <grlee77 at gmail.com> wrote:
>>
>>> Hi Gopi,
>>>
>>> I am not sure I follow exactly what you are suggesting here. Are you
>>> proposing that the `fftpackN` variable in your example would be a Python
>>> module corresponding to a given backend? If so, why is an array of data, z,
>>> being passed into the newAPI function here? I don't see why a data array
>>> should be needed at all to choose a backend.
>>>
>>> Please also see my response to Anany's recent post on this list for a
>>> couple of other suggestions of things to consider in terms of the API.
>>>
>>> - Greg
>>>
>>>
>>> On Fri, Mar 29, 2019 at 5:27 AM Gopi Manohar Tatiraju <
>>> deathcoderx at gmail.com> wrote:
>>>
>>>> Hey,
>>>> I went through the code and got some ideas.
>>>> So what will happen is:
>>>> First, we will declare FFTW as we use them normally.
>>>>
>>>> z = np.random.rand(2, 128, 64)
>>>> pyfftw.FFTW(z, axes=(-1, ))
>>>> ..
>>>> ..
>>>> Now after declaring our basic work to shift to another FFTpack, what we
>>>> can do is:
>>>> fftPackN= newAPI(z, pack = 'PYFFTW');
>>>>
>>>> Pack will take the name of the 3rd party FFT pack which we want to use.
>>>> Suggestions needed.
>>>>
>>>> Cheers.
>>>>
>>>>
>>>
>>>>
>>>> On Wed, Mar 27, 2019 at 8:43 PM Eric Larson <larson.eric.d at gmail.com>
>>>> wrote:
>>>>
>>>>> After reading the related issues and thinking about the problem,
>>>>> perhaps the best first step is you come up with some initial API proposal.
>>>>>
>>>>> One way to work through the process is to think about the use cases
>>>>> that we want to support. How should users actually write code to use the
>>>>> new functionality? You can start with short code snippets that do not work
>>>>> currently, but should work for users at the end of the project. Then think
>>>>> about what needs to change inside SciPy for these code snippets to work.
>>>>> This can become an iterative process as you realize that the SciPy changes
>>>>> might be better or easier with changes to the user-facing API, etc.
>>>>>
>>>>> Eric
>>>>>
>>>>>
>>>>> On Tue, Mar 26, 2019 at 3:38 PM Gopi Manohar Tatiraju <
>>>>> deathcoderx at gmail.com> wrote:
>>>>>
>>>>>> Hey,
>>>>>> As proposal submission period started I want to discuss a bit anobt
>>>>>> how can we implement the API.
>>>>>> I would very much appriciate inputs and guidance.
>>>>>>
>>>>>> Cheers.
>>>>>>
>>>>>> On Thu, Mar 7, 2019 at 11:40 PM Eric Larson <larson.eric.d at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> The best way to figure that out is probably to take a look at some
>>>>>>> of the existing (minor) SciPy GitHub issues and try to work on one or two
>>>>>>> that you feel like working on. Actually trying to work out how to
>>>>>>> contribute code changes (as independently as possible/reasonable based on
>>>>>>> the existing guides/docs already discoverable online) will quickly
>>>>>>> highlight things that need to be learned. Some of the general things are
>>>>>>> the SciPy contributing guidelines, standard GitHub PR workflow,
>>>>>>> communication with reviewers, and so forth.
>>>>>>>
>>>>>>> For this particular project, it's important to understand the
>>>>>>> various flavors and properties of FFT in SciPy (real, complex,
>>>>>>> n-dimensional, caching plans, etc.) as well as other projects like NumPy,
>>>>>>> CuPy, and pyFFTW. Some of this can be learned and improved upon as part of
>>>>>>> GSoC, but the more you know and can demonstrate / put into practice
>>>>>>> knowledge of before the applications are reviewed, the better.
>>>>>>>
>>>>>>> Eric
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Mar 6, 2019 at 4:55 AM Gopi Manohar Tatiraju <
>>>>>>> deathcoderx at gmail.com> wrote:
>>>>>>>
>>>>>>>> Thank you for the detailed reply.
>>>>>>>> I will keep these point in mind while preparing my proposal. Apart
>>>>>>>> from proposal, what skills shall I improve so that I can contribute more
>>>>>>>> effectively and efficiently.
>>>>>>>>
>>>>>>>> Cheers.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Mar 6, 2019, 3:42 AM Eric Larson <larson.eric.d at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> There are a number of considerations that would be good to
>>>>>>>>> address. I'd start by considering the things that are typically considered
>>>>>>>>> when making changes to SciPy
>>>>>>>>> <https://docs.scipy.org/doc/scipy/reference/hacking.html#contributing-to-scipy>,
>>>>>>>>> but I'd expand it to a longer list since the scope is bigger than a typical
>>>>>>>>> single PR. The list would contain (at least):
>>>>>>>>>
>>>>>>>>> - what will the user-facing API / contract be?
>>>>>>>>> - what in SciPy needs to change to allow it (internals /
>>>>>>>>> implementation outline, perhaps skeleton functions/classes)?
>>>>>>>>> - what examples should we show users (in general this ends up
>>>>>>>>> being just simple use cases of the API)?
>>>>>>>>> - how will we test it for correctness?
>>>>>>>>> - how will we benchmark it to prevent performance regressions?
>>>>>>>>> - how can we break up the work for these steps (if possible) into
>>>>>>>>> separable, sequential (or -- better but not usually possible --
>>>>>>>>> independent) PRs?
>>>>>>>>>
>>>>>>>>> You might notice that none of these points are actually all that
>>>>>>>>> specific to the FFT project, but can be generally applicable to GSoC
>>>>>>>>> projects. For the FFT project, I would say "how do we build the backend" is
>>>>>>>>> indeed implicitly contained / outlined in these bullet points. And as with
>>>>>>>>> most projects, the first two points (and possibly the last) will probably
>>>>>>>>> be the most challenging to sort out.
>>>>>>>>>
>>>>>>>>> Eric
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Mar 4, 2019 at 10:09 PM Gopi Manohar Tatiraju <
>>>>>>>>> deathcoderx at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hey Eric,
>>>>>>>>>> So for my proposal, shall I include how can we built a back end?
>>>>>>>>>> What all points shall I include to make it effective?
>>>>>>>>>>
>>>>>>>>>> Cheers.
>>>>>>>>>>
>>>>>>>>>> On Tue, Mar 5, 2019, 4:50 AM Eric Larson <larson.eric.d at gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> That would be me -- don't worry, I'm on this list, as are
>>>>>>>>>>> (presumably) other developers interested in the discussion, so no need to
>>>>>>>>>>> document elsewhere for now from our end. It looks like Greg has summarized
>>>>>>>>>>> the current status and thoughts we've had, so I don't have much to add at
>>>>>>>>>>> this point.
>>>>>>>>>>>
>>>>>>>>>>> Eric
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Sun, Mar 3, 2019 at 11:59 PM Gopi Manohar Tatiraju <
>>>>>>>>>>> deathcoderx at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hey Greg,
>>>>>>>>>>>> I got the basic idea about what we want to implement this year.
>>>>>>>>>>>> Our first priority is to create a backend for 3rd party
>>>>>>>>>>>> fftpacks. We can create API using Python and Flask or any other suitable
>>>>>>>>>>>> framework.
>>>>>>>>>>>>
>>>>>>>>>>>> I thought this discussion was going on on SciPy Developer List
>>>>>>>>>>>> only. So do I need to document it all and post it somewhere so that other
>>>>>>>>>>>> contributors can see this?
>>>>>>>>>>>> Also, I couldn't find any contact info regarding Eric. It would
>>>>>>>>>>>> be very helpful if we include him in this discussion as this is a roadmap
>>>>>>>>>>>> for GSoC project and I can also get more insight into the project and
>>>>>>>>>>>> expected outcomes.
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers.
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Mar 4, 2019 at 9:31 AM Gregory Lee <grlee77 at gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sun, Mar 3, 2019 at 12:28 PM Gopi Manohar Tatiraju <
>>>>>>>>>>>>> deathcoderx at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hey Gregory,
>>>>>>>>>>>>>> I went through the idea and I am very much interested in
>>>>>>>>>>>>>> working on it. I had Fourier Transforms as a part of my academic course, so
>>>>>>>>>>>>>> I am well aware with basic concepts.
>>>>>>>>>>>>>> One thing I want to discuss is, its mentioned that we have to
>>>>>>>>>>>>>> create a backend system for 3rd party FFTpacks so my dobuts
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1. How are we going to create API?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> This will needed to be decided on in consensus with the SciPy
>>>>>>>>>>>>> developers and user community. I am mentoring for the project as a
>>>>>>>>>>>>> contributor here and a maintainer of the related pyFFTW package, but I am
>>>>>>>>>>>>> not a core SciPy dev (the other mentor, Eric, is a core dev here). I have
>>>>>>>>>>>>> not discussed the API in any detail with the SciPy team at this stage.
>>>>>>>>>>>>> Working out details of what the API should look like and working to
>>>>>>>>>>>>> finalize that in collaboration with the community will be a primary task
>>>>>>>>>>>>> during the early part of the GSOC project.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2. After the creation of backend will user at the time of
>>>>>>>>>>>>>> installation of SciPy will decide what all 3rd party FFTpacks he needs or
>>>>>>>>>>>>>> we will include some by default?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The idea is to allow users to opt-in to a different FFT
>>>>>>>>>>>>> backend at run time, but these alternative libraries will not be
>>>>>>>>>>>>> distributed with SciPy itself due to incompatible software licenses (pyFFTW
>>>>>>>>>>>>> and mkl-fft) or a requirement for specific hardware (CuPy requires an
>>>>>>>>>>>>> NVIDIA GPU). SciPy will continue to offer its own bundled FFT library
>>>>>>>>>>>>> (currently fftpack, but this could be switched to pocketfft as discussed in
>>>>>>>>>>>>> the ideas page). The purpose of the backend interface is to provide a
>>>>>>>>>>>>> convenient way for users to have third party libraries perform the FFTs
>>>>>>>>>>>>> instead, but under the existing scipy.fftpack API. This is motivated by a
>>>>>>>>>>>>> desire by end users and downstream projects to have features such as
>>>>>>>>>>>>> multi-threading (pyFFTW and mkl-fft) or GPU support (CuPy) that are not
>>>>>>>>>>>>> implemented in scipy.fftpack. Ultimately it will be up to users that opt-in
>>>>>>>>>>>>> to using these other libraries to confirm that the license terms and are
>>>>>>>>>>>>> compatible with their use case.
>>>>>>>>>>>>>
>>>>>>>>>>>>> p.s. It is not a big deal, but I think the SciPy-Dev mailing
>>>>>>>>>>>>> lists prefers replies **below** the email you are replying to so people can
>>>>>>>>>>>>> more easily follow the thread of discussion. There are some guidelines at:
>>>>>>>>>>>>> https://www.scipy.org/scipylib/mailing-lists.html
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I will research more about how to implement a backend myself
>>>>>>>>>>>>>> and will let you know as soon as I come up with something new.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>> CHeers,
>>>>>>>>>>>>>> Greg
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Sun, Mar 3, 2019 at 10:25 PM Gregory Lee <
>>>>>>>>>>>>>> grlee77 at gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Sun, Mar 3, 2019 at 4:19 AM Gopi Manohar Tatiraju <
>>>>>>>>>>>>>>> deathcoderx at gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I want to work on SciPy in GSoC 2019. I started fixing some
>>>>>>>>>>>>>>>> bugs, also made some pull request. Can anyone suggest me how should I
>>>>>>>>>>>>>>>> proceed further?
>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>> SciPy-Dev mailing list
>>>>>>>>>>>>>>>> SciPy-Dev at python.org
>>>>>>>>>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Gopi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think you have probably already found it, but the GSOC
>>>>>>>>>>>>>>> project we have already identified mentors for in SciPy is here:
>>>>>>>>>>>>>>> https://github.com/scipy/scipy/wiki/GSoC-2019-project-ideas
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If this is a project you are interested in then I would
>>>>>>>>>>>>>>> start thinking about what the FFT interface would look like and reviewing
>>>>>>>>>>>>>>> some of the past/present issues related to fftpack / FFT in SciPy. Please
>>>>>>>>>>>>>>> ask questions you have related to that project here so that you can clarify
>>>>>>>>>>>>>>> what would be involved in writing a successful application.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If you have a different idea in mind you can propose it
>>>>>>>>>>>>>>> here, but you should do that soon, because it will be necessary to
>>>>>>>>>>>>>>> determine if there is interest from the SciPy team before you spend time
>>>>>>>>>>>>>>> coming up with a detailed proposal. We have to determine if a project
>>>>>>>>>>>>>>> sounds feasible to complete for GSOC and if appropriate mentors can be
>>>>>>>>>>>>>>> identified.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ultimately, you will need to submit an application to SciPy
>>>>>>>>>>>>>>> between March 25th - April 9th.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>> Greg
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>> SciPy-Dev mailing list
>>>>>>>>>>>>>>> SciPy-Dev at python.org
>>>>>>>>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> SciPy-Dev mailing list
>>>>>>>>>>>>>> SciPy-Dev at python.org
>>>>>>>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> SciPy-Dev mailing list
>>>>>>>>>>>>> SciPy-Dev at python.org
>>>>>>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> SciPy-Dev mailing list
>>>>>>>>>>>> SciPy-Dev at python.org
>>>>>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> SciPy-Dev mailing list
>>>>>>>>>>> SciPy-Dev at python.org
>>>>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> SciPy-Dev mailing list
>>>>>>>>>> SciPy-Dev at python.org
>>>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> SciPy-Dev mailing list
>>>>>>>>> SciPy-Dev at python.org
>>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> SciPy-Dev mailing list
>>>>>>>> SciPy-Dev at python.org
>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> SciPy-Dev mailing list
>>>>>>> SciPy-Dev at python.org
>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>>>
>>>>>> _______________________________________________
>>>>>> SciPy-Dev mailing list
>>>>>> SciPy-Dev at python.org
>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>>
>>>>> _______________________________________________
>>>>> SciPy-Dev mailing list
>>>>> SciPy-Dev at python.org
>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>
>>>> _______________________________________________
>>>> SciPy-Dev mailing list
>>>> SciPy-Dev at python.org
>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>
>> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20190402/1bdbe585/attachment-0001.html>