On Sun, Apr 28, 2019 at 5:41 PM Marten van Kerkwijk <m.h.vankerkwijk@gmail.com> wrote:
Hi Ralf,

Thanks for the comments and summary slides. I think you're over-interpreting my wish to break people's code! I certainly believe - and think we all agree - that we remain as committed as ever to ensure that
continues to work just as before. My main comment is that I want to ensure that no similar guarantee will exist for
(or whatever we call it). I think that is quite consistent with NEP-18, since as originally written there was not even the possibility to access the implementation directly (which was after long discussions about whether to allow it, including ideas like `import numpy.internal_apt as np`). In this respect, the current proposal is a large deviation from the original intent, so we need to be clear about what we are promising.

In summary, I think the guarantees should be as follows:
1.If you call np.function and
  - do not define __array_function__, changes happen only via the usual cycle.
  - define __array_function__, you take responsibility for returning the result.
2. If you call np.function.__wrapped__ and
  - input only ndarray, changes happen only via the usual cycle;
  - input anything but ndarray, changes can happen in any release.

Thanks. These guarantees make sense I think. __wrapped__ is new, so specifying that it should be called only with ndarrays seems reasonable.

On the larger picture,in your slides, the further split that happens is that if no override is present, the first thing that actually gets called is not the function implementation but rather `ndarray.__array_function__`. I think it is important to add this to your mental image (and the slides),

This is tricky. Yes, that's how it's implemented, but making __array_function__ do anything at all is potentially hazardous. Conceptually I think it's meant to be a do-nothing wrapper that just forwards directly to the actual wrapped function. Although if we could get it consistent, it may work.

since it means that generic parts of the implementations (like coercion ;-), can be moved there.

If all wrapped functions have identical generic parts, because there's just one ndarray.__array_function__. Are you sure that all functions have identical generic parts? I'm not - it's a bit of work, but I'd be surprised if there is even one line of code that's common between all functions. And if you have to special case functions within ndarray.__array_function__ to let some have asarray and some asanyarray etc., there's probably also a significant amount of overhead added.
There's also the thought of exposing the dispatching mechanism itself in the NEP: https://www.numpy.org/neps/nep-0018-array-function-protocol.html#use-outside-of-numpy. That may also get more complicated.

By the way, can you think of any other generic part besides coercion? There's other things that could be nice to skip (input validation for one) but nothing is generic.


For ufuncs, this is relatively easy, for other functions less so since they differ quite a bit in what coercion they do.

All the best,

NumPy-Discussion mailing list