On Sat, Apr 27, 2019 at 7:05 PM Marten van Kerkwijk < m.h.vankerkwijk@gmail.com> wrote:
Hi All,
I agree with Ralf that there are two discussions going on, but also with Hameer that they are related, in that part of the very purpose of __array_function__ was to gain freedom to experiment with implementations.
This may have been your motivation, but I don't see that in NEP 18 anywhere. On the contrary - the single sentence under "backwards compatibility" in the NEP reads: "This proposal does not change existing semantics, except for those arguments that currently have __array_function__ methods, which should be rare." If I'm missing something that's actually in NEP 18, can you please point out the actual text? By the way, I suspect that Hameer may have meant replacing __array_function__ completely (not sure though). Technically that may even be a good idea, but despite the "experimental" label I'm pretty sure that that ship has sailed given the review process we went through for NEP 18 and it starting to get uptake in other libraries. And in particular the freedom to *assume* that inputs are arrays so that we
can stop worrying about breaking subclasses and duck arrays: with __array_function__, we can now (well, eventually) tell people to just override the function and make it do what they want.
This is not in NEP 18, and against our backwards compatibility policy. In that respect, if having `__numpy_implementation__` means it would not
change the existing situation with backwards compatibility, then that is bad: we do want to change that! The point is to keep the exposed API stable, not the implementation.
API and semantics. The NEP really is very clear on this.
Of course, the idea proposed here is not to keep the implementation stable, but rather to help people implement __array_function__ who are daunted by the many functions that can be overridden, especially for those classes for which many numpy functions work out of the box: they can have a default of just calling the old implementation.
This is not feasible. All the things you talk about are related to NEP 22 (duck typing for NumPy arrays). NEP 18 is about overrides. It's not about skipping parts of the implementation of functions (like asarray). For code like: ``` import numpy as np import mylib x = mylib.customarray([1, 2, 3]) # duck array or subclass np.somefunc(x) ``` you simply cannot bypass asarray in the future if somefunc has asarray now. To get somefunc-without-asarray, you need a new opt-in interface. Telling our whole userbase to use __array_ufunc__ or __array_function__ to keep their code working is just not a good idea.
In the end, the proposed goal for __numpy_implementation__ really seems simple: to provide a better name for __wrapped__. But to me the discussion has proven that this is in fact not a good idea. It does suggest stability where there should be none, and the name itself is contentious. Maybe it is best to just stick with __wrapped__? If so, the the only change would be that we mention it in the NEP,
Sticking with __wrapped__ is fine with me too. It's a reasonable name. Marten, I understand where you're coming from and why you want to bypass array coercion, but this insistence on being able to break the world isn't helpful. I like the goal and would like to see a design for how to achieve it. It just needs that: a new design. It could be more protocols, or be based on uarray (what Hameer proposes, probably the better option because it's a coherent design that can do what you want), or yet something else. It seems like we all have a different mental model of what NEP 18 actually does. I'm going to try to put mine on a few slides with diagrams/examples to see if that helps, since mailing list threads are hard to process. Cheers, Ralf