[Numpy-discussion] Adding to the non-dispatched implementation of NumPy methods

[Ralf Gommers]

On Mon, Apr 22, 2019 at 9:26 PM Nathaniel Smith <njs at pobox.com> wrote:

> Your last email didn't really clarify anything for me. I get that
> np.func.__numpy_implementation__ is intended to have the semantics of
> numpy's implementation of func, but that doesn't tell me much :-). And
> also, that's exactly the definition of np.func, isn't it?
>
> You're talking about ~doubling the size of numpy's API,
>

I think we can already get both the NEP 18 wrapped functions and their
underlying implementations today, based on the value of
NUMPY_EXPERIMENTAL_ARRAY_FUNCTION.

It looks to me like all this proposed change does is bypass a
do-very-little wrapper.

and don't seem able to even articulate what the new API's commitments are.
> This still makes me nervous. Maybe it should have a NEP? What's your
> testing strategy for all the new functions?
>

The current decorator mechanism already checks that the signatures match,
so it shouldn't be possible to get a mismatch. So probably not much is
needed beyond some assert_equal(np.func(...),
np.func.__numpy_implementation__(...)) checks.

@Stephan the PR for the NEP change is very hard to parse. Maybe easier to
just open a PR with an implementation for one or a few functions +
associated tests?

Cheers,
Ralf



> On Mon, Apr 22, 2019, 09:22 Stephan Hoyer <shoyer at gmail.com> wrote:
>
>> Are there still concerns here? If not, I would love to move ahead with
>> these changes so we can get this into NumPy 1.17.
>>
>> On Tue, Apr 16, 2019 at 10:23 AM Stephan Hoyer <shoyer at gmail.com> wrote:
>>
>>> __numpy_implementation__ is indeed simply a slot for third-parties to
>>> access NumPy's implementation. It should be considered "NumPy's current
>>> implementation", not "NumPy's implementation as of 1.14". Of course, in
>>> practice these will remain very similar, because we are already very
>>> conservative about how we change NumPy.
>>>
>>> I would love to have clean well-defined coercion semantics for every
>>> NumPy function, which would be implicitly adopted by
>>> `__numpy_implementation__` (e.g., we could say that every function always
>>> coerces its arguments with `np.asarray()`). But I think that's an
>>> orthogonal issue. We have been supporting some ad-hoc duck typing in NumPy
>>> for a long time (e.g., the `.sum()` method which is called by `np.sum()`).
>>> Removing that would require a deprecation cycle, which may indeed be
>>> warranted once we're sure we're happy with __array_function__. But I don't
>>> think the deprecation cycle will be any worse if the implementation is also
>>> exposed via `__numpy_implementation__`.
>>>
>>> We should definitely still think about a cleaner "core" implementation
>>> of NumPy functions in terms of a minimal core. One recent example of this
>>> can be found JAX (see
>>> https://github.com/google/jax/blob/04b45e4086249bad691a33438e8bb6fcf639d001/jax/numpy/lax_numpy.py).
>>> This would be something appropriate to put into a more generic function
>>> attribute on NumPy functions, perhaps `__array_implementation__`. But I
>>> don't think formalizing `__numpy_implementation__` as a way to get access
>>> to NumPy's default implementation will limit our future options here.
>>>
>>> Cheers,
>>> Stephan
>>>
>>>
>>> On Tue, Apr 16, 2019 at 6:44 AM Marten van Kerkwijk <
>>> m.h.vankerkwijk at gmail.com> wrote:
>>>
>>>>
>>>> I somewhat share Nathaniel's worry that by providing
>>>> `__numpy_implementation__` we essentially get stuck with the
>>>> implementations we have currently, rather than having the hoped-for freedom
>>>> to remove all the `np.asarray` coercion. In that respect, an advantage of
>>>> using `_wrapped` is that it is clearly a private method, so anybody is
>>>> automatically forewarned that this can change.
>>>>
>>>> In principle, ndarray.__array_function__ would be more logical, but as
>>>> noted in the PR, the problem is that it is non-trivial for a regular
>>>> __array_function__ implementation to coerce all the arguments to ndarray
>>>> itself.
>>>>
>>>> Which suggests that perhaps what is missing is a general routine that
>>>> does that, i.e., that re-uses the dispatcher.
>>>>
>>>> -- Marten
>>>> _______________________________________________
>>>> NumPy-Discussion mailing list
>>>> NumPy-Discussion at python.org
>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>
>>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190422/1172ca6c/attachment.html>