[Numpy-discussion] Adding to the non-dispatched implementation of NumPy methods

[Nathaniel Smith]

Your last email didn't really clarify anything for me. I get that
np.func.__numpy_implementation__ is intended to have the semantics of
numpy's implementation of func, but that doesn't tell me much :-). And
also, that's exactly the definition of np.func, isn't it?

You're talking about ~doubling the size of numpy's API, and don't seem able
to even articulate what the new API's commitments are. This still makes me
nervous. Maybe it should have a NEP? What's your testing strategy for all
the new functions?

On Mon, Apr 22, 2019, 09:22 Stephan Hoyer <shoyer at gmail.com> wrote:

> Are there still concerns here? If not, I would love to move ahead with
> these changes so we can get this into NumPy 1.17.
>
> On Tue, Apr 16, 2019 at 10:23 AM Stephan Hoyer <shoyer at gmail.com> wrote:
>
>> __numpy_implementation__ is indeed simply a slot for third-parties to
>> access NumPy's implementation. It should be considered "NumPy's current
>> implementation", not "NumPy's implementation as of 1.14". Of course, in
>> practice these will remain very similar, because we are already very
>> conservative about how we change NumPy.
>>
>> I would love to have clean well-defined coercion semantics for every
>> NumPy function, which would be implicitly adopted by
>> `__numpy_implementation__` (e.g., we could say that every function always
>> coerces its arguments with `np.asarray()`). But I think that's an
>> orthogonal issue. We have been supporting some ad-hoc duck typing in NumPy
>> for a long time (e.g., the `.sum()` method which is called by `np.sum()`).
>> Removing that would require a deprecation cycle, which may indeed be
>> warranted once we're sure we're happy with __array_function__. But I don't
>> think the deprecation cycle will be any worse if the implementation is also
>> exposed via `__numpy_implementation__`.
>>
>> We should definitely still think about a cleaner "core" implementation of
>> NumPy functions in terms of a minimal core. One recent example of this can
>> be found JAX (see
>> https://github.com/google/jax/blob/04b45e4086249bad691a33438e8bb6fcf639d001/jax/numpy/lax_numpy.py).
>> This would be something appropriate to put into a more generic function
>> attribute on NumPy functions, perhaps `__array_implementation__`. But I
>> don't think formalizing `__numpy_implementation__` as a way to get access
>> to NumPy's default implementation will limit our future options here.
>>
>> Cheers,
>> Stephan
>>
>>
>> On Tue, Apr 16, 2019 at 6:44 AM Marten van Kerkwijk <
>> m.h.vankerkwijk at gmail.com> wrote:
>>
>>>
>>> I somewhat share Nathaniel's worry that by providing
>>> `__numpy_implementation__` we essentially get stuck with the
>>> implementations we have currently, rather than having the hoped-for freedom
>>> to remove all the `np.asarray` coercion. In that respect, an advantage of
>>> using `_wrapped` is that it is clearly a private method, so anybody is
>>> automatically forewarned that this can change.
>>>
>>> In principle, ndarray.__array_function__ would be more logical, but as
>>> noted in the PR, the problem is that it is non-trivial for a regular
>>> __array_function__ implementation to coerce all the arguments to ndarray
>>> itself.
>>>
>>> Which suggests that perhaps what is missing is a general routine that
>>> does that, i.e., that re-uses the dispatcher.
>>>
>>> -- Marten
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at python.org
>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>
>> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190422/e8c3057f/attachment-0001.html>