[Numpy-discussion] NEP 37: A dispatch protocol for NumPy-like modules

Ralf Gommers ralf.gommers at gmail.com
Fri Apr 10 06:27:28 EDT 2020


On Fri, Apr 10, 2020 at 5:17 AM Sebastian Berg <sebastian at sipsolutions.net>
wrote:

> On Thu, 2020-04-09 at 13:52 +0200, Ralf Gommers wrote:
> > On Thu, Apr 9, 2020 at 12:02 AM Sebastian Berg <
> > sebastian at sipsolutions.net>
> > wrote:
> >
> <snip>
> > >
> >
> > I think it would be nice to have a separate NEP 37 implementation
> > outside
> > of NumPy to play with. Unlike __array_function__, I don't think it
> > has to
> > go into NumPy immediately. This avoids the whole "experimental API"
> > issue,
>
> Fair enough, I have created a hopefully working start here:
>
> https://github.com/seberg/numpy_dispatch
>
> (this is not tested much at all yet, so it could be very buggy).
>

Thanks!


> There are a couple of additional features that I added.
>
> 1. A global opt-in (it is impossible to opt-out once opted in!)
> 2. A local opt-in (to guarantee opt-in if global flag is not set)
> 3. I added features to allow transitioning::
>
>       get_array_module(*arrays, modules="numpy",
>             future_modules=("dask.array", "cupy"), fallback="warn")
>
>    Will give FutureWarning/DeprecationWarning where necessary, in the
>    above "numpy" is supported, dask and cupy are supported but not
>    enabled by default. `None` works to say "all modules".
>    Once the transition is done, just move dask and cupy into `modules`
>    and remove `fallback=None`.
>

So future_modules explicitly excludes compatible libraries that are not
listed. Why would you want anyone to do that? I don't understand "supported
but not enabled", and it looks undesirable to me to special-case any
library in this mechanism.

Cheers,
Ralf


4. If there are FutureWarnings/DeprecationWarnigs the user needs to be
>    able to opt-in to future behaviour. Opting out can be done by
>    casting inputs. Opting-in is done using::
>
>       with future_dispatch_behavior():
>           call_library_function()
>
> Obviously, we may not want these features, but I was curious how we
> could provide the tools to allow clean transitions.
>
> Both context managers should be thread-safe, but I did not test that.
>
> The best try would probably be cupy and sklearn again, so I will give a
> ping on the sklearn PR. To make that easier, I tried to hack a bit of a
> "util" to allow testing (please scroll down on the readme on github).
>
> Best,
>
> Sebastian
>
>
>
> > it would be quite useful to test this with, e.g., CuPy + scikit-learn
> > without being stuck with any decisions in a released NumPy version.
> > Also
> > makes switching on/off very easy for users, just (don't) `pip install
> > numpy-array-module`.
> >
> > Cheers,
> > Ralf
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20200410/907f0fe6/attachment.html>


More information about the NumPy-Discussion mailing list