Re: [Numpy-discussion] Using nditer + external_loop to Always Iterate by Column
I am trying to use the nditer to traverse each column of a 2D array, returning the column as a 1D array. Consulting the docs, I found this example which works perfectly fine: In [65]: a = np.arange(6).reshape(2,3) In [66]: for x in np.nditer(a, flags=['external_loop'], order='F'): ...: print(x, end=' ') ...: [0 3] [1 4] [2 5] When changing the shape of the input array to (1, 3) however, this doesn’t yield what I am hoping for any more (essentially [0], [1] [2]): In [68]: for x in np.nditer(a, flags=['external_loop'], order='F'): ...: print(x, end=' ') ...: [0 1 2] I suspect this may have to do with the fact that the (1, 3) array is both C and F contiguous, and it is trying to return as large of a 1D F-contiguous array as it can. However, I didn’t see any way to really force it to go by columns. My best guess was the itershape argument though I couldn’t figure out how to get that to work and didn’t see much in the documentation. Thanks in advance for the help! - Will
Hi William, You can simply use a for loop for that task: Python 3.8.2 | packaged by conda-forge | (default, Apr 24 2020, 07:56:27) Type 'copyright', 'credits' or 'license' for more information IPython 7.14.0 -- An enhanced Interactive Python. Type '?' for help. In [1]: import numpy as np In [2]: a = np.arange(3).reshape((1, 3)) In [3]: for x in a.T: ...: print(x) ...: [0] [1] [2] Best regards, Hameer Abbasi From: NumPy-Discussion <numpy-discussion-bounces+einstein.edison=gmail.com@python.org> on behalf of William Ayd <william.ayd@icloud.com> Reply to: Discussion of Numerical Python <numpy-discussion@python.org> Date: Tuesday, 19. May 2020 at 01:42 To: "numpy-discussion@python.org" <numpy-discussion@python.org> Subject: Re: [Numpy-discussion] Using nditer + external_loop to Always Iterate by Column I am trying to use the nditer to traverse each column of a 2D array, returning the column as a 1D array. Consulting the docs, I found this example which works perfectly fine: In [65]: a = np.arange(6).reshape(2,3) In [66]: for x in np.nditer(a, flags=['external_loop'], order='F'): ...: print(x, end=' ') ...: [0 3] [1 4] [2 5] When changing the shape of the input array to (1, 3) however, this doesn’t yield what I am hoping for any more (essentially [0], [1] [2]): In [68]: for x in np.nditer(a, flags=['external_loop'], order='F'): ...: print(x, end=' ') ...: [0 1 2] I suspect this may have to do with the fact that the (1, 3) array is both C and F contiguous, and it is trying to return as large of a 1D F-contiguous array as it can. However, I didn’t see any way to really force it to go by columns. My best guess was the itershape argument though I couldn’t figure out how to get that to work and didn’t see much in the documentation. Thanks in advance for the help! - Will
Hi Will, To force an iteration to run along certain axes, I believe you should be using `op_axes`. Your diagnosis is correct that `external_loop` is trying to help you be more optimal, since it's purpose is exactly that: optimization. Unfortunately, if you use `op_axes` you'll run into https://github.com/numpy/numpy/issues/9808. Eric On Tue, 19 May 2020 at 00:42, William Ayd <william.ayd@icloud.com> wrote:
I am trying to use the nditer to traverse each column of a 2D array, returning the column as a 1D array. Consulting the docs, I found this example which works perfectly fine:
In [*65*]: a = np.arange(6).reshape(2,3)
In [*66*]: *for* x *in* np.nditer(a, flags=['external_loop'], order='F'): ...: print(x, end=' ') ...:
[0 3] [1 4] [2 5]
When changing the shape of the input array to (1, 3) however, this doesn’t yield what I am hoping for any more (essentially [0], [1] [2]):
In [*68*]: *for* x *in* np.nditer(a, flags=['external_loop'], order='F'): ...: print(x, end=' ') ...:
[0 1 2]
I suspect this may have to do with the fact that the (1, 3) array is both C and F contiguous, and it is trying to return as large of a 1D F-contiguous array as it can. However, I didn’t see any way to really force it to go by columns. My best guess was the *itershape* argument though I couldn’t figure out how to get that to work and didn’t see much in the documentation.
Thanks in advance for the help!
- Will
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
On Tue, 2020-05-19 at 17:11 +0100, Eric Wieser wrote:
Hi Will,
To force an iteration to run along certain axes, I believe you should be using `op_axes`. Your diagnosis is correct that `external_loop` is trying to help you be more optimal, since it's purpose is exactly that: optimization.
Unfortunately, if you use `op_axes` you'll run into https://github.com/numpy/numpy/issues/9808.
Yeah, I do not think what you want is possible with nditer. `op_axes` allows you to ignore certain axis, but as Eric points out, that actually ignores that axis entirely. So it acts similar to slicing the whole axis away. I previously thought it could be nice to have a new `arr.iteraxis(axis=None)` command. As a base, it would act like a flat iter, otherwise iterate the axis listed. Maybe with an additional flag whether its allowed to optimize iteration order (although from the python side I expect we do not want to do that by default). The reason is also that I am not sure I like `arr.flat` and `for subarr in arr` too much, because the list-of-list like iteration seems only semi-natural for an N-D array. - Sebastian
Eric
On Tue, 19 May 2020 at 00:42, William Ayd <william.ayd@icloud.com> wrote:
I am trying to use the nditer to traverse each column of a 2D array, returning the column as a 1D array. Consulting the docs, I found this example which works perfectly fine:
In [*65*]: a = np.arange(6).reshape(2,3)
In [*66*]: *for* x *in* np.nditer(a, flags=['external_loop'], order='F'): ...: print(x, end=' ') ...:
[0 3] [1 4] [2 5]
When changing the shape of the input array to (1, 3) however, this doesn’t yield what I am hoping for any more (essentially [0], [1] [2]):
In [*68*]: *for* x *in* np.nditer(a, flags=['external_loop'], order='F'): ...: print(x, end=' ') ...:
[0 1 2]
I suspect this may have to do with the fact that the (1, 3) array is both C and F contiguous, and it is trying to return as large of a 1D F-contiguous array as it can. However, I didn’t see any way to really force it to go by columns. My best guess was the *itershape* argument though I couldn’t figure out how to get that to work and didn’t see much in the documentation.
Thanks in advance for the help!
- Will
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
participants (4)
-
Eric Wieser -
Hameer Abbasi -
Sebastian Berg -
William Ayd