Giving deprecation of e.g. `float(np.array([1]))` a shot (not 0-d)
Hi all, Unlike conversions of 0-d arrays via: float(np.array([1])) conversions of 1-D or higher dimensional arrays with a single element are a bit strange: float(np.array([1])) And deprecating it has come up often enough with many in favor, but also many worried about the possible annoyance to users. I decided to give the PR a shot, I may have misread the room on it though: https://github.com/numpy/numpy/pull/10615 So if this turns out noisy (or you may simply disagree), I am happy to revert! There was always the worry that it might be painful for downstream. SciPy, pandas, matplotlib should all be fine (were fixed in the past years). And the fact that SciPy required much more changes than the other gives me some hope that many libraries won't mind. For end-users, I would lean towards taking it slow, but if you see issues there we can also revert of course. Cheers, Sebastian
On Thu, Apr 20, 2023 at 9:12 AM Sebastian Berg
Hi all,
Unlike conversions of 0-d arrays via:
float(np.array([1]))
conversions of 1-D or higher dimensional arrays with a single element are a bit strange:
float(np.array([1]))
And deprecating it has come up often enough with many in favor, but also many worried about the possible annoyance to users. I decided to give the PR a shot, I may have misread the room on it though:
https://github.com/numpy/numpy/pull/10615
So if this turns out noisy (or you may simply disagree), I am happy to revert!
This looks like a great clean-up to me, thanks for giving this a try!
If symmetry w.r.t. pytorch is any guide, it was nice to have it:
In [38]: float(torch.as_tensor([2]))
Out[38]: 2.0
In [39]: float(np.asarray([2]))
Out[39]: 2.0
I guess this boils down to what is a scalar really: is it `scalar.size
== 1` or `scalar.ndim == 0` or something else.
But that's just a digression, nevermind.
On Thu, Apr 20, 2023 at 7:25 PM Stephan Hoyer
On Thu, Apr 20, 2023 at 9:12 AM Sebastian Berg
wrote: Hi all,
Unlike conversions of 0-d arrays via:
float(np.array([1]))
conversions of 1-D or higher dimensional arrays with a single element are a bit strange:
float(np.array([1]))
And deprecating it has come up often enough with many in favor, but also many worried about the possible annoyance to users. I decided to give the PR a shot, I may have misread the room on it though:
https://github.com/numpy/numpy/pull/10615
So if this turns out noisy (or you may simply disagree), I am happy to revert!
This looks like a great clean-up to me, thanks for giving this a try! _______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: evgeny.burovskiy@gmail.com
On Thu, Apr 20, 2023 at 12:39 PM Evgeni Burovski
If symmetry w.r.t. pytorch is any guide, it was nice to have it:
In [38]: float(torch.as_tensor([2])) Out[38]: 2.0
In [39]: float(np.asarray([2])) Out[39]: 2.0
My question would be: Did they have a positive use case for this behavior, or were they just reflecting NumPy's behavior? AFAICR, the main reasoning on our side was that there was an unambiguous value that we _could_ return, so we might as well. And in our later experience, it was more trouble than it was worth. -- Robert Kern
On 4/20/23, Sebastian Berg
Hi all,
Unlike conversions of 0-d arrays via:
float(np.array([1]))
conversions of 1-D or higher dimensional arrays with a single element are a bit strange:
float(np.array([1]))
And deprecating it has come up often enough with many in favor, but also many worried about the possible annoyance to users. I decided to give the PR a shot, I may have misread the room on it though:
https://github.com/numpy/numpy/pull/10615
So if this turns out noisy (or you may simply disagree), I am happy to revert!
There was always the worry that it might be painful for downstream. SciPy, pandas, matplotlib should all be fine (were fixed in the past years). And the fact that SciPy required much more changes than the other gives me some hope that many libraries won't mind.
For end-users, I would lean towards taking it slow, but if you see issues there we can also revert of course.
Cheers,
Sebastian
Thanks Nico, and Sebastian, and everyone else involved in the PRs. This also affects `np.float64`: ``` In [61]: np.__version__ Out[61]: '1.25.0.dev0+1203.g1acac891f' In [62]: np.float64(0.0) Out[62]: 0.0 In [63]: np.float64(np.array(0.0)) Out[63]: 0.0 In [64]: np.float64(np.array([0.0])) <ipython-input-64-0f0309f2cf0c>:1: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.) np.float64(np.array([0.0])) Out[64]: 0.0 In [65]: np.float64(np.array([0.0, 0.0])) Out[65]: array([0., 0.]) ``` In 1.24.2, `np.float64(np.array([0.0])` returns the the scalar 0.0. If passing arrays to `np.float64()` is intentionally supported, it seems it would be more consistent for `np.float64(np.array([0.0]))` to return `np.array([0.0])`. That is how the other numpy types work (e.g. `np.complex128`, `np.int64`, etc.). But I'm not sure if there is a deprecation/update path that would get us there. Warren
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: warren.weckesser@gmail.com
On Thu, 2023-04-20 at 13:59 -0400, Warren Weckesser wrote:
On 4/20/23, Sebastian Berg
wrote: Hi all,
<snip>
In [64]: np.float64(np.array([0.0])) <ipython-input-64-0f0309f2cf0c>:1: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.) np.float64(np.array([0.0])) Out[64]: 0.0
In [65]: np.float64(np.array([0.0, 0.0])) Out[65]: array([0., 0.])
Hmmmpf, that would be a good follow-up to fix. In theory a FutureWarning I guess (returning the array), but in practice, I think we should just give the correct array result. (I don't love returning arrays from scalar constructors, but that is another thing and not for now.) - Sebsatian
```
In 1.24.2, `np.float64(np.array([0.0])` returns the the scalar 0.0.
If passing arrays to `np.float64()` is intentionally supported, it seems it would be more consistent for `np.float64(np.array([0.0]))` to return `np.array([0.0])`. That is how the other numpy types work (e.g. `np.complex128`, `np.int64`, etc.). But I'm not sure if there is a deprecation/update path that would get us there.
Warren
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: warren.weckesser@gmail.com
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
On Thu, 2023-04-20 at 20:17 +0200, Sebastian Berg wrote:
On Thu, 2023-04-20 at 13:59 -0400, Warren Weckesser wrote:
On 4/20/23, Sebastian Berg
wrote: Hi all,
<snip>
In [64]: np.float64(np.array([0.0])) <ipython-input-64-0f0309f2cf0c>:1: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.) np.float64(np.array([0.0])) Out[64]: 0.0
In [65]: np.float64(np.array([0.0, 0.0])) Out[65]: array([0., 0.])
Do you have any thoughts on how to make progress Warren? Had a bit of a look at it. You are probably aware that this is because for float, str, and bytes (our subclasses of them), we have (approximately): def __new__(cls, *args, **kwargs): try: super().__new__(*args, **kwargs) except: if len(args) != 1 or kwargs != {}: raise return np.asarray(args[0])[()] # scalar if 0-D For float64, I am tempted to just remove the super() path entirely and put in a fast-path for simple scalar object (like python `int`, `float`, `bool`, `str`) to avoid the full `np.asarray()` call. For unicode/bytes its a bit of a mess though? I suspect for them the `array` path is currently just useless in practice, because even arrays are interpreted as scalars here. The best path might be even to just deprecate array input entirely for them? Even then you have at least one case that is tricky: np.bytes_(5) returns an empty string (since we strip zeros) but if we would do the same as `np.asarray(5, dtype=np.bytes_)[()]` we would get a different result. (And raising on a non 0-D array doesn't help there.) Maybe the right way is to go as far and check if both paths match for non-trivial bytes?! - Sebastian
Hmmmpf, that would be a good follow-up to fix. In theory a FutureWarning I guess (returning the array), but in practice, I think we should just give the correct array result.
(I don't love returning arrays from scalar constructors, but that is another thing and not for now.)
- Sebsatian
```
In 1.24.2, `np.float64(np.array([0.0])` returns the the scalar 0.0.
If passing arrays to `np.float64()` is intentionally supported, it seems it would be more consistent for `np.float64(np.array([0.0]))` to return `np.array([0.0])`. That is how the other numpy types work (e.g. `np.complex128`, `np.int64`, etc.). But I'm not sure if there is a deprecation/update path that would get us there.
Warren
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: warren.weckesser@gmail.com
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
On 4/21/23, Sebastian Berg
On Thu, 2023-04-20 at 20:17 +0200, Sebastian Berg wrote:
On Thu, 2023-04-20 at 13:59 -0400, Warren Weckesser wrote:
On 4/20/23, Sebastian Berg
wrote: Hi all,
<snip>
In [64]: np.float64(np.array([0.0])) <ipython-input-64-0f0309f2cf0c>:1: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.) np.float64(np.array([0.0])) Out[64]: 0.0
In [65]: np.float64(np.array([0.0, 0.0])) Out[65]: array([0., 0.])
Do you have any thoughts on how to make progress Warren?
Sorry for the late reply; the recent comment in https://github.com/numpy/numpy/issues/23400 reminded me of this. As noted in the link in the recent comment in that issue, handling of nonscalar inputs of the numpy scalar types was also briefly discussed in the mailing list three years ago: https://mail.python.org/pipermail/numpy-discussion/2020-April/080566.html I don't have any concrete ideas other than outright deprecating the handling of anything that is not a scalar, but that might be too disruptive. Warren
Had a bit of a look at it. You are probably aware that this is because for float, str, and bytes (our subclasses of them), we have (approximately):
def __new__(cls, *args, **kwargs): try: super().__new__(*args, **kwargs) except: if len(args) != 1 or kwargs != {}: raise
return np.asarray(args[0])[()] # scalar if 0-D
For float64, I am tempted to just remove the super() path entirely and put in a fast-path for simple scalar object (like python `int`, `float`, `bool`, `str`) to avoid the full `np.asarray()` call.
For unicode/bytes its a bit of a mess though? I suspect for them the `array` path is currently just useless in practice, because even arrays are interpreted as scalars here.
The best path might be even to just deprecate array input entirely for them? Even then you have at least one case that is tricky:
np.bytes_(5)
returns an empty string (since we strip zeros) but if we would do the same as `np.asarray(5, dtype=np.bytes_)[()]` we would get a different result. (And raising on a non 0-D array doesn't help there.)
Maybe the right way is to go as far and check if both paths match for non-trivial bytes?!
- Sebastian
Hmmmpf, that would be a good follow-up to fix. In theory a FutureWarning I guess (returning the array), but in practice, I think we should just give the correct array result.
(I don't love returning arrays from scalar constructors, but that is another thing and not for now.)
- Sebsatian
```
In 1.24.2, `np.float64(np.array([0.0])` returns the the scalar 0.0.
If passing arrays to `np.float64()` is intentionally supported, it seems it would be more consistent for `np.float64(np.array([0.0]))` to return `np.array([0.0])`. That is how the other numpy types work (e.g. `np.complex128`, `np.int64`, etc.). But I'm not sure if there is a deprecation/update path that would get us there.
Warren
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: warren.weckesser@gmail.com
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: warren.weckesser@gmail.com
On Tue, 2023-05-16 at 11:46 -0400, Warren Weckesser wrote:
On 4/21/23, Sebastian Berg
wrote: On Thu, 2023-04-20 at 20:17 +0200, Sebastian Berg wrote:
On Thu, 2023-04-20 at 13:59 -0400, Warren Weckesser wrote:
On 4/20/23, Sebastian Berg
wrote: Hi all,
<snip>
In [64]: np.float64(np.array([0.0])) <ipython-input-64-0f0309f2cf0c>:1: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.) np.float64(np.array([0.0])) Out[64]: 0.0
In [65]: np.float64(np.array([0.0, 0.0])) Out[65]: array([0., 0.])
Do you have any thoughts on how to make progress Warren?
Sorry for the late reply; the recent comment in https://github.com/numpy/numpy/issues/23400 reminded me of this. As noted in the link in the recent comment in that issue, handling of nonscalar inputs of the numpy scalar types was also briefly discussed in the mailing list three years ago: https://mail.python.org/pipermail/numpy-discussion/2020-April/080566.html
I don't have any concrete ideas other than outright deprecating the handling of anything that is not a scalar, but that might be too disruptive.
Not sure it is very disruptive, but I don't have a lot of appetite for a full deprecation myself right now. Was more hoping to nail down the best (now) solution that isn't a deprecation :). - Sebastian
Warren
Had a bit of a look at it. You are probably aware that this is because for float, str, and bytes (our subclasses of them), we have (approximately):
def __new__(cls, *args, **kwargs): try: super().__new__(*args, **kwargs) except: if len(args) != 1 or kwargs != {}: raise
return np.asarray(args[0])[()] # scalar if 0-D
For float64, I am tempted to just remove the super() path entirely and put in a fast-path for simple scalar object (like python `int`, `float`, `bool`, `str`) to avoid the full `np.asarray()` call.
For unicode/bytes its a bit of a mess though? I suspect for them the `array` path is currently just useless in practice, because even arrays are interpreted as scalars here.
The best path might be even to just deprecate array input entirely for them? Even then you have at least one case that is tricky:
np.bytes_(5)
returns an empty string (since we strip zeros) but if we would do the same as `np.asarray(5, dtype=np.bytes_)[()]` we would get a different result. (And raising on a non 0-D array doesn't help there.)
Maybe the right way is to go as far and check if both paths match for non-trivial bytes?!
- Sebastian
Hmmmpf, that would be a good follow-up to fix. In theory a FutureWarning I guess (returning the array), but in practice, I think we should just give the correct array result.
(I don't love returning arrays from scalar constructors, but that is another thing and not for now.)
- Sebsatian
```
In 1.24.2, `np.float64(np.array([0.0])` returns the the scalar 0.0.
If passing arrays to `np.float64()` is intentionally supported, it seems it would be more consistent for `np.float64(np.array([0.0]))` to return `np.array([0.0])`. That is how the other numpy types work (e.g. `np.complex128`, `np.int64`, etc.). But I'm not sure if there is a deprecation/update path that would get us there.
Warren
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: warren.weckesser@gmail.com
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: warren.weckesser@gmail.com
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
participants (5)
-
Evgeni Burovski
-
Robert Kern
-
Sebastian Berg
-
Stephan Hoyer
-
Warren Weckesser