On Sat, Sep 7, 2019 at 5:08 PM Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Sat, Sep 7, 2019 at 4:16 PM Nathaniel Smith <njs@pobox.com> wrote:
On Fri, Sep 6, 2019 at 11:04 PM Ralf Gommers <ralf.gommers@gmail.com> wrote:
Vendoring means "include the code". So no dependency on an external package. If we don't vendor, it's going to be either unused, or end up as a dependency for the whole SciPy/PyData stack.
If we vendor it then it also ends up as a dependency for the whole SciPy/PyData stack...
It seems you're just using an unusual definition here. Dependency == a package you have to install, is present in pyproject.toml/install_requires, shows up in https://github.com/numpy/numpy/network/dependencies, etc.
That's a pretty trivial definition though. Surely the complexity of the installed code and its maintainer structure is what matters, not the exact details of how the install happens.
Actually, now that we've discussed the fft issue, I'd suggest to change the NEP to: vendor, and make default for fft, random, and linalg.
There's no way we can have an effective discussion of duck arrays, fft backends, random backends, and linalg backends all at once in a single thread.
Can you write separate NEPs for each of these? Some questions I'd like to see addressed:
For fft: - fft is an entirely self-contained operation, with no interactions with the rest of the system; the only difference between implementations is speed. What problems are caused by monkeypatching,
It was already explained in this thread, it's been on our roadmap for ~2 years at least, and monkeypatching is pretty much universally understood to be bad. If that's not enough, please search the NumPy issues for "monkeypatching". You'll find issues like https://github.com/numpy/numpy/issues/12374#issuecomment-438725645. At the moment this is very confusing, and hard to diagnose - you have to install a whole new NumPy and then find that the problem is gone (or not). Being able to switch backends in one line of code and re-test would be very valuable.
Sure, it's not meant a trick question, I'm just saying you should write down the reasons and how you solve them in one place. Maybe some of the reasons monkeypatching is bad don't apply here, or maybe some of them do apply, but uarray doesn't solve them – we can't tell without doing the work. The link you gave doesn't involve monkeypatching or np.fft, so I'm not sure how it's relevant...?
It seems perhaps more useful to have a call so we can communicate with higher bandwidth, rather than lots of writing new NEPs here? In preparation, we need to write up in more detail how __array_function__ and unumpy fit together, rather than treat different pieces all separately (because the problems and pros/cons really won't change much between functions and submodules). I'll defer answering your other questions till that's done, so the discussion is hopefully a bit more structured.
I don't have a lot of time for calls, and you'd still have to write it up for everyone who isn't on the call... -n -- Nathaniel J. Smith -- https://vorpus.org