[Numpy-discussion] Adding to the non-dispatched implementation of NumPy methods

Stephan Hoyer shoyer at gmail.com
Mon Apr 15 19:38:38 EDT 2019

On Mon, Apr 15, 2019 at 1:21 PM Nathaniel Smith <njs at pobox.com> wrote:

> What's the difference between
> np.concatenate.__numpy_implementation__(...)
> and
> np.ndarray.__array_function__(np.concatenate, ...)
> ?

I can answer this technically, though this doesn't seem to be quite what
you're looking for:
- The former always succeed at dispatch, because it coerces all arguments
to NumPy arrays.
- The second will either return NotImplemented (if a non-NumPy arrays
implements __array_function__), or give the same result as former.

> More generally, I guess I'm not quite clear on how to think about what the
> "no dispatch" version does, because obviously it doesn't make sense to have
> *no* dispatch. Instead it's something like "the legacy hard-coded dispatch"?

__numpy_implementation__ means you skip __array_function__ dispath and call
the original NumPy function. In practice, this means you get legacy
hard-coded dispatch behavior in most cases, e.g., the result will always be
in the form of NumPy array(s).

It doesn't mean that the implementation always coerces all arguments to
NumPy arrays. For example, np.result_type() will pull out of .dtype
attributes off of its arguments, even without necessarily coercing its
arguments to NumPy arrays. This strange version of "the implementation for
NumPy arrays" turns out to be something that several libraries that want to
implement __array_function__ want to be able to continue to use on their
own array objects (namely, JAX and CuPy).

> On Mon, Apr 15, 2019, 08:30 Stephan Hoyer <shoyer at gmail.com> wrote:
>> Hi everyone,
>> We have a proposed a revision to NEP-18 (__array_function__). The
>> proposal is for an adding an alias to the non-dispatched version of NumPy
>> array functions in the __numpy_implementaiton__ function attribute:
>> https://github.com/numpy/numpy/pull/13305
>> I believe this attribute improves the protocol in three ways:
>> 1. It provides a hook that __array_function__ methods can use to call
>> implementation intended for NumPy arrays. This allows for "partial
>> implementations" of NumPy's API, which turns out to useful even for some
>> array libraries that reimplement nearly everything (namely, for CuPy and
>> JAX).
>> 2. It allows for fast access to the non-dispatching version of NumPy
>> functions, e.g.,
>> np.concatenate.__numpy_implementation__(list_of_all_numpy_arrays).
>> 3. Internally, the implementation of numpy.ndarray.__array_function__ now
>> looks identical to how we encourage outside developers to write their own
>> __array_function__ methods. The dispatching logic no longer includes a
>> special case for NumPy arrays.
>> Feedback would be greatly welcomed!
>> Best,
>> Stephan
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190415/24811f4e/attachment-0001.html>

More information about the NumPy-Discussion mailing list