On Mon, Apr 22, 2019 at 9:26 PM Nathaniel Smith <njs@pobox.com> wrote:

Your last email didn't really clarify anything for me. I get that np.func.__numpy_implementation__ is intended to have the semantics of numpy's implementation of func, but that doesn't tell me much :-). And also, that's exactly the definition of np.func, isn't it?

You're talking about ~doubling the size of numpy's API,

I think we can already get both the NEP 18 wrapped functions and their underlying implementations today, based on the value of NUMPY_EXPERIMENTAL_ARRAY_FUNCTION. It looks to me like all this proposed change does is bypass a do-very-little wrapper. and don't seem able to even articulate what the new API's commitments are.

This still makes me nervous. Maybe it should have a NEP? What's your testing strategy for all the new functions?

The current decorator mechanism already checks that the signatures match, so it shouldn't be possible to get a mismatch. So probably not much is needed beyond some assert_equal(np.func(...), np.func.__numpy_implementation__(...)) checks. @Stephan the PR for the NEP change is very hard to parse. Maybe easier to just open a PR with an implementation for one or a few functions + associated tests? Cheers, Ralf

On Mon, Apr 22, 2019, 09:22 Stephan Hoyer <shoyer@gmail.com> wrote:

Are there still concerns here? If not, I would love to move ahead with these changes so we can get this into NumPy 1.17.

On Tue, Apr 16, 2019 at 10:23 AM Stephan Hoyer <shoyer@gmail.com> wrote:

__numpy_implementation__ is indeed simply a slot for third-parties to access NumPy's implementation. It should be considered "NumPy's current implementation", not "NumPy's implementation as of 1.14". Of course, in practice these will remain very similar, because we are already very conservative about how we change NumPy.

I would love to have clean well-defined coercion semantics for every NumPy function, which would be implicitly adopted by `__numpy_implementation__` (e.g., we could say that every function always coerces its arguments with `np.asarray()`). But I think that's an orthogonal issue. We have been supporting some ad-hoc duck typing in NumPy for a long time (e.g., the `.sum()` method which is called by `np.sum()`). Removing that would require a deprecation cycle, which may indeed be warranted once we're sure we're happy with __array_function__. But I don't think the deprecation cycle will be any worse if the implementation is also exposed via `__numpy_implementation__`.

We should definitely still think about a cleaner "core" implementation of NumPy functions in terms of a minimal core. One recent example of this can be found JAX (see https://github.com/google/jax/blob/04b45e4086249bad691a33438e8bb6fcf639d001/...). This would be something appropriate to put into a more generic function attribute on NumPy functions, perhaps `__array_implementation__`. But I don't think formalizing `__numpy_implementation__` as a way to get access to NumPy's default implementation will limit our future options here.

Cheers, Stephan

On Tue, Apr 16, 2019 at 6:44 AM Marten van Kerkwijk < m.h.vankerkwijk@gmail.com> wrote:

I somewhat share Nathaniel's worry that by providing `__numpy_implementation__` we essentially get stuck with the implementations we have currently, rather than having the hoped-for freedom to remove all the `np.asarray` coercion. In that respect, an advantage of using `_wrapped` is that it is clearly a private method, so anybody is automatically forewarned that this can change.

In principle, ndarray.__array_function__ would be more logical, but as noted in the PR, the problem is that it is non-trivial for a regular __array_function__ implementation to coerce all the arguments to ndarray itself.

Which suggests that perhaps what is missing is a general routine that does that, i.e., that re-uses the dispatcher.

-- Marten _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion