[Numpy-discussion] Adding to the non-dispatched implementation of NumPy methods

[Nathaniel Smith]

On Sun, Apr 28, 2019 at 1:38 PM Marten van Kerkwijk
<m.h.vankerkwijk at gmail.com> wrote:
>
> Hi Nathaniel,
>
> I'm a bit confused why` np.concatenate([1, 2], [3, 4])` would be a problem. In the current model, all (numpy) functions fall back to `ndarray.__array_function__`, which does know what to do with anything that doesn't have `__array_function__`: it just coerces it to array. Am I missing something?

IMO, the reason that having ndarray.__array_function__ was attractive
in the first place, was that we were hoping it would let you pretend
that there's nothing special about ndarray.  Like, when you call
np.concatenate, it just looks for __array_function__ methods and
dispatches to them; sometimes that means calling
thirdpartyobject.__array_function__, and sometimes it means calling
ndarray.__array_function__, but as far as np.concatenate is concerned
those are interchangeable and treated in the same way.

But in fact ndarray.__array_function__ *is* special. I guess you could
write down the semantics so that np.concatenate([1, 2], [3, 4]) still
calls ndarray.__array_function__, by defining a special dispatch rules
just for ndarray.__array_function__. But if ndarray.__array_function__
isn't going to follow the same dispatch rule, then why should it exist
and be called "__array_function__"? A special method like
"__array_function__" is nothing except a name for a dispatch rule.

And if we add __skipping_array_function__, it makes this even worse.
In a model where dispatch always goes through *some* object's
__array_function__, then __skipping_array_function__ makes no sense --
if you skip __array_function__ then there's nothing left.

You might try to save it by saying, ok, we'll only skip third-party
__array_function__, but still dispatch to ndarray.__array_function__.
But this doesn't work either.
np.concatenate.__skipping_array_function__(...) is different from
ndarray.__array_function__(np.concatenate, ...), because they treat
arguments with __array_function__ methods differently. (The former
coerces them to ndarray; the latter returns NotImplemented.) Neither
can be implemented in terms of the other (!).

ndarray.__array_function__ was a nice idea, but I don't think there's
any way to fit into a coherent system.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org