On Thu, 2020-04-09 at 13:52 +0200, Ralf Gommers wrote:
On Thu, Apr 9, 2020 at 12:02 AM Sebastian Berg < sebastian@sipsolutions.net> wrote:
<snip>
I think it would be nice to have a separate NEP 37 implementation outside of NumPy to play with. Unlike __array_function__, I don't think it has to go into NumPy immediately. This avoids the whole "experimental API" issue,
Fair enough, I have created a hopefully working start here: https://github.com/seberg/numpy_dispatch (this is not tested much at all yet, so it could be very buggy). There are a couple of additional features that I added. 1. A global opt-in (it is impossible to opt-out once opted in!) 2. A local opt-in (to guarantee opt-in if global flag is not set) 3. I added features to allow transitioning:: get_array_module(*arrays, modules="numpy", future_modules=("dask.array", "cupy"), fallback="warn") Will give FutureWarning/DeprecationWarning where necessary, in the above "numpy" is supported, dask and cupy are supported but not enabled by default. `None` works to say "all modules". Once the transition is done, just move dask and cupy into `modules` and remove `fallback=None`. 4. If there are FutureWarnings/DeprecationWarnigs the user needs to be able to opt-in to future behaviour. Opting out can be done by casting inputs. Opting-in is done using:: with future_dispatch_behavior(): call_library_function() Obviously, we may not want these features, but I was curious how we could provide the tools to allow clean transitions. Both context managers should be thread-safe, but I did not test that. The best try would probably be cupy and sklearn again, so I will give a ping on the sklearn PR. To make that easier, I tried to hack a bit of a "util" to allow testing (please scroll down on the readme on github). Best, Sebastian
it would be quite useful to test this with, e.g., CuPy + scikit-learn without being stuck with any decisions in a released NumPy version. Also makes switching on/off very easy for users, just (don't) `pip install numpy-array-module`.
Cheers, Ralf