On Sun, Apr 28, 2019 at 9:20 PM Stephan Hoyer <shoyer@gmail.com> wrote:
On Sun, Apr 28, 2019 at 8:42 AM Marten van Kerkwijk <m.h.vankerkwijk@gmail.com> wrote:
In summary, I think the guarantees should be as follows:
1.If you call np.function and
  - do not define __array_function__, changes happen only via the usual cycle.
  - define __array_function__, you take responsibility for returning the result.
2. If you call np.function.__wrapped__ and
  - input only ndarray, changes happen only via the usual cycle;
  - input anything but ndarray, changes can happen in any release.

The uses that I've seen so far (in CuPy and JAX), involve a handful of functions that are directly re-exported from NumPy, e.g., jax.numpy.array_repr is the exact same object as numpy.array_repr:

I suspect this will be less common in the future if __array_function__ takes off, but for now it's convenient because users don't need to know exactly which functions have been reimplemented. They can just use "import jax.numpy as np" and everything works.

These libraries are indeed passing CuPy or JAX arrays into NumPy functions, which currently happen to have the desired behavior, thanks to accidental details about how NumPy currently supports duck-typing and/or coercions.

To this end, it would be really nice to have an alias that *is* guaranteed to work exactly as if __array_function__ didn't exist, and not only for numpy.ndarray arrays.

Just to be clear: for this purpose, being able to call the implementation is still mostly a convenient crutch, correct? For classes that define __array_function__, would you expect more than the guarantee I wrote above, that the wrapped version will continue to work as advertised for ndarray input only?

In particular, suppose we change an implementation to use different other numpy functions inside (which are of course overridden using __array_function__). I could imagine situations where  that would work fine for everything that does not define __array_ufunc__, but where it would not for classes that do define it. Is that then a problem for numpy or for the project that has a class that defines __array_function__?
Ralf raises a good point about the name. We don't need to add this attribute for ufuncs and __array_ufunc__ yet, but
(1) we might want this in the future, just for consistency in the design of __array_function__ and __array_ufunc__, and
(2) we definitely don't want to rule out converting functions into ufunc.

So we might as well pick a name that works for both, e.g., __skip_array_overrides__ rather than __skip_array_function__. This would let us save our users a bit of pain by not requiring them to make changes like  np.where.__skip_array_function__ -> np.where.__skip_array_ufunc__.

Note that for ufuncs it is not currently possible to skip the override. I don't think it is super-hard to do it, but I'm not sure I see the need to add a crutch where none has been needed so far. More generally, it is not obvious there is any C code where skipping the override is useful, since the C code relies much more directly on inputs being ndarray.
All the best,


NumPy-Discussion mailing list