[SciPy-Dev] Backend support for fftpack - GSoC 2019

[Peter Bell]

>> Should backends be allowed to add functionality or should they be completely
>> interchangeable. E.g. unlike fftpack, FFTW has support for long doubles;
>> should this be exposed to SciPy users?
>
> If a user writes backend-specific code, then they may just as well bypass the
> backend right, and call the other library directly?

I agree, in fact I quote you in the proposal saying something similar.

> Although for long double things will work out of the box perhaps, because the
> dtype of an input array doesn't come back in the API. So the array can be
> passed straight to the backend.

This was why I picked it as an example. It doesn't add any new parameters or
change the function signature at all, but it does change the result of a
particular call to the API. So it's a very minimal change yet it still breaks
the interchangeability of backends.
> This is probably also a good time to point out http://uarray.readthedocs.io/.
> It can provide a complete backend system without too much work

After reading through the docs it does seem to cover this and more. However, it
also seems to be unstable at this point so I’m not sure it’s wise to rely on it just
yet.

> it was designed exactly for the type of problem you're trying to solve here.

Unless I'm misunderstanding it seems to be more focused on supporting different
types of array, rather than different implementations for NumPy arrays. I can't
see any utility in having a complicated dispatch mechanism when we really only
want to have a single backend at a time for any given function.

I would assume that because of this, uarray would come with a greater
performance overhead. Whereas I think the overhead of an fftpack backend should
just be some argument validation and a single function call. I could always
implement it both ways and benchmark it if there's interest.

Peter

From: SciPy-Dev <scipy-dev-bounces+peterbell10=live.co.uk at python.org> On Behalf Of Ralf Gommers
Sent: 08 May 2019 23:59
To: SciPy Developers List <scipy-dev at python.org>
Subject: Re: [SciPy-Dev] Backend support for fftpack - GSoC 2019



On Wed, May 8, 2019 at 5:50 PM Peter Bell <PeterBell10 at live.co.uk<mailto:PeterBell10 at live.co.uk>> wrote:
Hello all,

I’m happy to say my GSoC proposal was accepted,

Hey Peter, congratulations! We're happy to have you on board!

so I’ll be working over the summer to add support for multiple fftpack backends. For those interested, I’ve extracted the main discussion from my proposal which you can read here:
https://docs.google.com/document/d/1hGMc6ZrsmSi_LSixI5oFbo5cZkPUo92sfmnVthg1u6Y/edit

Thanks, interesting. Will take a little while to process. I just give some very initial thoughts below (so take with a grain of salt).


There are a several design decisions I raise in the proposal and any comments would be appreciated. Of particular note:
* Should backends be allowed to add functionality or should they be completely interchangeable. E.g. unlike fftpack, FFTW has support for long doubles; should this be exposed to SciPy users?

If a user writes backend-specific code, then they may just as well bypass the backend right, and call the other library directly? Although for long double things will work out of the box perhaps, because the dtype of an input array doesn't come back in the API. So the array can be passed straight to the backend.

* Is there any interest in adding a SciPy config file? For just one option it’s probably not worthwhile but if there is interest in more config options then it starts to make more sense.

My sense is no. Something like a context manager (`with backend('pyfftw'):`) is lightweight enough probably.

This is probably also a good time to point out http://uarray.readthedocs.io/. It can provide a complete backend system without too much work, it was designed exactly for the type of problem you're trying to solve here. Would be good to have your thoughts on that.


Ideally a clear set of design goals can be agreed upon now, before it gets too far into the coding period which begins at the end of the month.

Yes great idea.

Cheers,
Ralf

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20190509/5ad15bb3/attachment-0001.html>