On Wed, 2020-04-08 at 17:04 -0400, Andreas Mueller wrote:
Hey all. Is there any update on this? Is there any input we can provide as users? I'm not entirely sure where you are in the decision making process right now :)
Hey, thanks for the ping. Things are a bit stuck right now. I think what we need is some clarity on the implications and alternatives. I was thinking about organizing a small conference call with the main people interested in the next weeks. There are also still some alternatives to this NEP in the race, and we may need to clarify which ones are actually still in the race... Maybe to see some of the possible sticking points: 1. What do we do about SciPy, have it under this umbrella? And how would we want to design that. 2. Context managers have some composition issues, maybe less so if they are in the downstream package. Or should we have global defaults as well? 3. How do we ensure safe transitions for users as much as possible. * If you use this, can functions suddenly return a different type in the future? * Should we force you to cast to NumPy arrays in a transition period, or force you to somehow silence a transition warning? 4. Is there a serious push to have a "reduced" API or even a versioned API? But I am probably forgetting some other things. In my personal opinion, I think NEP 37 with minor modifications is still the best duck in the race. I feel we should be able to find a reasonable solution for SciPy. Point 2. about Context managers may be true, but this is much smaller in scope from the ones uarray proposed IIRC, and I could not figure out major scoping issues with it yet (the sklearn draft). About the safe transition, that may be the stickiest point. But e.g. if you enable `get_array_module` sklearn could limit a certain function to error out if it finds something other than NumPy? The main problem is how to do opt-in into future behaviour. A context manager can do that, although the danger is that someone just uses that everywhere... On the reduced/versioned API front, I would hope that we can defer that as a semi-orthogonal issue, basically saying that for now you have to provide a NumPy API that faithfully reproduces whatever NumPy version is installed on the system. Cheers, Sebastian
Cheers, Andy
On 3/3/20 6:34 PM, Sebastian Berg wrote:
One of the main rationales for the whole NEP, and the argument in multiple places ( https://numpy.org/neps/nep-0037-array-module.html#opt-in-vs-opt-out-for-user... ) is that it's now opt-in while __array_function__ was opt-out. This isn't really true - the problem is simply *moved*, from the duck array libraries to the array-consuming libraries. The end user will still see the backwards incompatible change, with no way to turn it off. It will be easier with __array_module__ to warn users, but this should be expanded on in the NEP. Might it be possible to flip this NEP back to opt-out while keeping
On 2/23/20 6:59 PM, Ralf Gommers wrote: the nice simplifications and configurabile array-creation routines, relative to __array_function__?
That is, what if we define two modules, "numpy" and "numpy_strict". "numpy_strict" would raise an exception on duck-arrays defining __array_module__ (as numpy currently does). "numpy" would be a wrapper around "numpy_strict" that decorates all numpy methods with a call to "get_array_module(inputs).func(inputs)". This would be possible, but I think we strongly leaned against the idea. Basically, if you have to opt-out, from a library perspective
On Fri, 2020-02-28 at 11:28 -0500, Allan Haldane wrote: there may be `np.asarray` calls, which for example later call into C and expect arrays. So, I have large doubts that an opt-out solution works easily for library authors. Array function is opt-out, but effectively most clean library code already opted out...
We had previously discussed the opposite, of having a namespace of implicit dispatching based on get_array_module, but if we keep array function around, I am not sure there is much reason for it.
Then end-user code that did "import numpy as np" would accept ducktypes by default, while library developers who want to signal they don't support ducktypes can opt-out by doing "import numpy_strict as np". Issues with `np.as_array` seem mitigated compared to __array_function__ since that method would now be ducktype-aware. My tendency is that if we want to go there, we would need to push ahead with the `np.duckarray()` idea instead.
To be clear: I currently very much prefer the get_array_module() idea. It just seems much cleaner for library authors, and they are the primary issue at the moment in my opinion.
- Sebastian
-Allan _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion