[Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

Ralf Gommers ralf.gommers at gmail.com
Sat Sep 7 01:54:08 EDT 2019

On Fri, Sep 6, 2019 at 5:16 PM Nathaniel Smith <njs at pobox.com> wrote:

> On Fri, Sep 6, 2019 at 11:44 AM Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> >
> >
> >
> > On Fri, Sep 6, 2019 at 1:32 AM Hameer Abbasi <einstein.edison at gmail.com>
> wrote:
> >>
> >> That's a lot of very good questions! Let me see if I can answer them
> one-by-one.
> >>
> >> On 06.09.19 09:49, Nathaniel Smith wrote:
> >>
> >> But even that could be accomplished by just
> >> putting something in the docs. And adding the alias has substantial
> >> risks: it makes unumpy tied to the numpy release cycle and
> >> compatibility rules, and it means that we're committing to maintaining
> >> unumpy ~forever even if Hameer or Quansight move onto other things.
> >> That seems like a lot to take on for such vague benefits?
> >>
> >> I can assure you Travis has had the goal of "replatforming SciPy" from
> as far back as I met him, he's spawned quite a few efforts in that
> direction along with others from Quansight (and they've led to nice
> projects). Quansight, as I see it, is unlikely to abandon something like
> this if it becomes successful (and acceptance of this NEP will be a huge
> success story).
> >
> >
> > Let me address this separately, since it's not really a technical
> concern.
> >
> > First, this is not what we say for other contributions. E.g. we didn't
> say no to Pocketfft because Martin Reineck may move on, or
> __array_function__ because Stephan may get other interests at some point,
> or a whole new numpy.random, etc.
> >
> > Second, this is not about Quansight. At Quansight Labs we've been able
> to create time for Hameer to build this, and me and others to contribute -
> which is very nice, but the two are not tied inextricably together. In the
> end it's still individuals submitting this NEP. I have been a NumPy dev for
> ~10 years before joining Quansight, and my future NumPy contributions are
> not dependent on staying at Quansight (not that I plan to go anywhere!).
> I'm guessing the same is true for others.
> >
> > Third, unumpy is a fairly thin layer over uarray, which already has
> another user in SciPy.
> I'm sorry if that came across as some kind snipe at Quansight
> specifically. I didn't mean it that way. It's a much more general
> concern: software projects are inherently risky, and often fail;
> companies and research labs change focus and funding shifts around.
> This is just a general risk that we need to take that into account
> when making decisions. And when there are proposals to add new
> submodules to numpy, we always put them under intense scrutiny,
> exactly because of the support commitments.

Yes, that's fair, and we should be critical here. All code we accept is
indeed a maintenance burden.

> The new fft and random code are replacing/extending our existing
> public APIs that we already committed to, so that's a very different
> situation. And __array_function__ was something that couldn't work at
> all without being built into numpy, and even then it was controversial
> and merged on an experimental basis. It's always about trade-offs. My
> concern here is that the NEP is proposing that the numpy maintainers
> take on this large commitment,

Again, not just the NumPy maintainers. There really isn't that much in
`unumpy` that's all that complicated. And again, `uarray` has multiple
maintainers (note that Peter is also a SciPy core dev) and has another user
in SciPy.

*and* AFAICT there's no compensating
> benefit to justify that: everything that can be done with
> numpy.overridable can be done just as well with a standalone unumpy
> package... right?

True, mostly. But at that point, if we say that it's the way to do array
coercion, and creation (and perhaps some other things as well), we're
saying at the same time that every other package that needs this (e.g.
Dask, CuPy) should take unumpy as a hard dependency. Which is a much bigger
ask than when it comes with NumPy. We can discuss it of course.

Major exception is if we want to make it default for some functionality,
like for example numpy.fft (I'll answer your other email for that.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190906/6a011d36/attachment-0001.html>

More information about the NumPy-Discussion mailing list