[Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

Ralf Gommers ralf.gommers at gmail.com
Sat Sep 7 20:07:32 EDT 2019


On Sat, Sep 7, 2019 at 4:16 PM Nathaniel Smith <njs at pobox.com> wrote:

> On Fri, Sep 6, 2019 at 11:04 PM Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> >  Vendoring means "include the code". So no dependency on an external
> package. If we don't vendor, it's going to be either unused, or end up as a
> dependency for the whole SciPy/PyData stack.
>
> If we vendor it then it also ends up as a dependency for the whole
> SciPy/PyData stack...
>

It seems you're just using an unusual definition here. Dependency == a
package you have to install, is present in pyproject.toml/install_requires,
shows up in https://github.com/numpy/numpy/network/dependencies, etc.


> > Actually, now that we've discussed the fft issue, I'd suggest to change
> the NEP to: vendor, and make default for fft, random, and linalg.
>
> There's no way we can have an effective discussion of duck arrays, fft
> backends, random backends, and linalg backends all at once in a single
> thread.
>
> Can you write separate NEPs for each of these? Some questions I'd like
> to see addressed:
>
> For fft:
> - fft is an entirely self-contained operation, with no interactions
> with the rest of the system; the only difference between
> implementations is speed. What problems are caused by monkeypatching,
>

It was already explained in this thread, it's been on our roadmap for ~2
years at least, and monkeypatching is pretty much universally understood to
be bad. If that's not enough, please search the NumPy issues for
"monkeypatching". You'll find issues like
https://github.com/numpy/numpy/issues/12374#issuecomment-438725645. At the
moment this is very confusing, and hard to diagnose - you have to install a
whole new NumPy and then find that the problem is gone (or not). Being able
to switch backends in one line of code and re-test would be very valuable.

It seems perhaps more useful to have a call so we can communicate with
higher bandwidth, rather than lots of writing new NEPs here? In
preparation, we need to write up in more detail how __array_function__ and
unumpy fit together, rather than treat different pieces all separately
(because the problems and pros/cons really won't change much between
functions and submodules). I'll defer answering your other questions till
that's done, so the discussion is hopefully a bit more structured.

Cheers,
Ralf



and how is uarray materially different from monkeypatching?
>
> For random:
> - I thought the new random implementation with pluggable generators
> etc. was supposed to solve this problem already. Why doesn't it?
> - The biggest issue with MKL monkeypatching random is that it breaks
> stream stability. How does the uarray approach address this?
>
> For linalg:
> - linalg already support __array_ufunc__ for overrides. Why do we need
> a second override system? Isn't that redundant?
>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190907/149db8ab/attachment.html>


More information about the NumPy-Discussion mailing list