[Numpy-discussion] Adding to the non-dispatched implementation of NumPy methods

[Ralf Gommers]

On Sun, Apr 28, 2019 at 5:41 PM Marten van Kerkwijk <
m.h.vankerkwijk at gmail.com> wrote:

> Hi Ralf,
>
> Thanks for the comments and summary slides. I think you're
> over-interpreting my wish to break people's code! I certainly believe - and
> think we all agree - that we remain as committed as ever to ensure that
> ```
> np.function(inputs)
> ```
> continues to work just as before. My main comment is that I want to ensure
> that no similar guarantee will exist for
> ```
> np.function.__wrapped__(inputs)
> ```
> (or whatever we call it). I think that is quite consistent with NEP-18,
> since as originally written there was not even the possibility to access
> the implementation directly (which was after long discussions about whether
> to allow it, including ideas like `import numpy.internal_apt as np`). In
> this respect, the current proposal is a large deviation from the original
> intent, so we need to be clear about what we are promising.
>
> In summary, I think the guarantees should be as follows:
> 1.If you call np.function and
>   - do not define __array_function__, changes happen only via the usual
> cycle.
>   - define __array_function__, you take responsibility for returning the
> result.
> 2. If you call np.function.__wrapped__ and
>   - input only ndarray, changes happen only via the usual cycle;
>   - input anything but ndarray, changes can happen in any release.
>

Thanks. These guarantees make sense I think. __wrapped__ is new, so
specifying that it should be called only with ndarrays seems reasonable.


> On the larger picture,in your slides, the further split that happens is
> that if no override is present, the first thing that actually gets called
> is not the function implementation but rather `ndarray.__array_function__`.
> I think it is important to add this to your mental image (and the slides),
>

This is tricky. Yes, that's how it's implemented, but making
__array_function__ do anything at all is potentially hazardous.
Conceptually I think it's meant to be a do-nothing wrapper that just
forwards directly to the actual wrapped function. Although if we could get
it consistent, it may work.

since it means that generic parts of the implementations (like coercion
> ;-), can be moved there.
>

If all wrapped functions have identical generic parts, because there's just
one ndarray.__array_function__. Are you sure that all functions have
identical generic parts? I'm not - it's a bit of work, but I'd be surprised
if there is even one line of code that's common between all functions. And
if you have to special case functions within ndarray.__array_function__ to
let some have asarray and some asanyarray etc., there's probably also a
significant amount of overhead added.

There's also the thought of exposing the dispatching mechanism itself in
the NEP:
https://www.numpy.org/neps/nep-0018-array-function-protocol.html#use-outside-of-numpy.
That may also get more complicated.

By the way, can you think of any other generic part besides coercion?
There's other things that could be nice to skip (input validation for one)
but nothing is generic.

Cheers,
Ralf


For ufuncs, this is relatively easy, for other functions less so since they
> differ quite a bit in what coercion they do.
>
> All the best,
>
> Marten
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190428/104ff201/attachment.html>