[Numpy-discussion] Adding to the non-dispatched implementation of NumPy methods

Stephan Hoyer shoyer at gmail.com
Thu Apr 25 00:44:35 EDT 2019

On Tue, Apr 23, 2019 at 12:27 AM Nathaniel Smith <njs at pobox.com> wrote:

> On Mon, Apr 22, 2019 at 11:13 PM Stephan Hoyer <shoyer at gmail.com> wrote:
> >
> >> On Mon, Apr 22, 2019 at 9:26 PM Nathaniel Smith <njs at pobox.com> wrote:
> >>>
> >>> Your last email didn't really clarify anything for me. I get that
> np.func.__numpy_implementation__ is intended to have the semantics of
> numpy's implementation of func, but that doesn't tell me much :-). And
> also, that's exactly the definition of np.func, isn't it?
> >
> > My understanding of the protocol we came up with in NEP-18 is that every
> NumPy function (that takes array-like arguments) now has two parts to its
> implementation:
> > 1. The NEP-18 part involving calling the dispatcher function, and
> checking for/calling __array_function__ attributes on array-like arguments.
> This part is documented in NEP-18.
> > 2. The original function definition, which is called if either (a) no
> __array_function__ attributes exist, or (b) the only __array_function__
> attribute is numpy.ndarray.__array_function__. This part is documented in
> the docstring of the NumPy function.
> >
> > "__numpy_implementation__" provides a short-cut to (2) without (1).
> That's it.
> OK, so the semantics are: the same as the normal function, except we
> pretend that none of the arguments have an __array_function__
> attribute?
> That's much clearer to me than how you were phrasing it before :-).

OK, I will make sure something like this ends up in the NEP :)

> Though now the name "__numpy_implementation__" doesn't seem very
> evocative of what it does... numpy's dispatch sequence has changed a
> lot in the past (mostly adding new coercion rules), and will probably
> change in the future, and "__numpy_implementation__" doesn't give much
> guidance about which parts of the dispatch sequence should be skipped
> as "dispatch" and which should be included as "implementation". Maybe
> something like __skipping_array_function__?

With "__numpy_implementation__" I was hoping to evoke "the implementation
used by numpy.ndarray.__array_function__" and "the implementation for NumPy
arrays" rather than "the implementation found in the NumPy library." So it
would still be appropriate to use on functions defined in SciPy, as long as
they are defined on NumPy arrays.

That said, this is clearly going to remain a source of confusion. So let's
see if we can do better.

Taking a step back, there will be three generic parts to NumPy functions
after NEP-18:
1. Dispatching with __array_function__
2. Coercion to NumPy arrays (sometimes skipped if an object has the
necessary duck-typing methods)
3. Implementation (either in C or is terms of other NumPy functions/methods)

Currently, NumPy functions do steps (2) and (3) together. What we're asking
for here is a way to continue this behavior in the future, by optionally
skipping step (1). But in the future, as Marten notes below, we should not
rule out cases where we also want to skip straight to step (3), without
step (2).

"__skipping_array_function__"  would be a reasonable choice, though it does
not evoke the "numpy array specific"  aspect that I want to emphasis. Also,
it has the unfortunate aspect of being named after what it doesn't do,
rather than what it does.

"__numpy_ndarray_implementation__" and "__numpy_array_implementation__" are
a bit verbose, but maybe they would be better?

The generic "__wrapped__" seems like a pretty bad choice to me, both
because it's not at all descriptive and because it's generically used by
functools.wraps -- which means np.ndarray.__array_function__ could
inadvertently succeed when called with non-NumPy functions. Let's at least
stick to unique names for our protocols :).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190424/122afff8/attachment.html>

More information about the NumPy-Discussion mailing list