Re: [Numpydiscussion] Using nditer + external_loop to Always Iterate by Column
I am trying to use the nditer to traverse each column of a 2D array, returning the column as a 1D array. Consulting the docs, I found this example which works perfectly fine: In [65]: a = np.arange(6).reshape(2,3) In [66]: for x in np.nditer(a, flags=['external_loop'], order='F'): ...: print(x, end=' ') ...: [0 3] [1 4] [2 5] When changing the shape of the input array to (1, 3) however, this doesn’t yield what I am hoping for any more (essentially [0], [1] [2]): In [68]: for x in np.nditer(a, flags=['external_loop'], order='F'): ...: print(x, end=' ') ...: [0 1 2] I suspect this may have to do with the fact that the (1, 3) array is both C and F contiguous, and it is trying to return as large of a 1D Fcontiguous array as it can. However, I didn’t see any way to really force it to go by columns. My best guess was the itershape argument though I couldn’t figure out how to get that to work and didn’t see much in the documentation. Thanks in advance for the help!  Will
Hi William, You can simply use a for loop for that task: Python 3.8.2  packaged by condaforge  (default, Apr 24 2020, 07:56:27) Type 'copyright', 'credits' or 'license' for more information IPython 7.14.0  An enhanced Interactive Python. Type '?' for help. In [1]: import numpy as np In [2]: a = np.arange(3).reshape((1, 3)) In [3]: for x in a.T: ...: print(x) ...: [0] [1] [2] Best regards, Hameer Abbasi From: NumPyDiscussion <numpydiscussionbounces+einstein.edison=gmail.com@python.org> on behalf of William Ayd <william.ayd@icloud.com> Reply to: Discussion of Numerical Python <numpydiscussion@python.org> Date: Tuesday, 19. May 2020 at 01:42 To: "numpydiscussion@python.org" <numpydiscussion@python.org> Subject: Re: [Numpydiscussion] Using nditer + external_loop to Always Iterate by Column I am trying to use the nditer to traverse each column of a 2D array, returning the column as a 1D array. Consulting the docs, I found this example which works perfectly fine: In [65]: a = np.arange(6).reshape(2,3) In [66]: for x in np.nditer(a, flags=['external_loop'], order='F'): ...: print(x, end=' ') ...: [0 3] [1 4] [2 5] When changing the shape of the input array to (1, 3) however, this doesn’t yield what I am hoping for any more (essentially [0], [1] [2]): In [68]: for x in np.nditer(a, flags=['external_loop'], order='F'): ...: print(x, end=' ') ...: [0 1 2] I suspect this may have to do with the fact that the (1, 3) array is both C and F contiguous, and it is trying to return as large of a 1D Fcontiguous array as it can. However, I didn’t see any way to really force it to go by columns. My best guess was the itershape argument though I couldn’t figure out how to get that to work and didn’t see much in the documentation. Thanks in advance for the help!  Will
Hi Will, To force an iteration to run along certain axes, I believe you should be using `op_axes`. Your diagnosis is correct that `external_loop` is trying to help you be more optimal, since it's purpose is exactly that: optimization. Unfortunately, if you use `op_axes` you'll run into https://github.com/numpy/numpy/issues/9808. Eric On Tue, 19 May 2020 at 00:42, William Ayd <william.ayd@icloud.com> wrote:
I am trying to use the nditer to traverse each column of a 2D array, returning the column as a 1D array. Consulting the docs, I found this example which works perfectly fine:
In [*65*]: a = np.arange(6).reshape(2,3)
In [*66*]: *for* x *in* np.nditer(a, flags=['external_loop'], order='F'): ...: print(x, end=' ') ...:
[0 3] [1 4] [2 5]
When changing the shape of the input array to (1, 3) however, this doesn’t yield what I am hoping for any more (essentially [0], [1] [2]):
In [*68*]: *for* x *in* np.nditer(a, flags=['external_loop'], order='F'): ...: print(x, end=' ') ...:
[0 1 2]
I suspect this may have to do with the fact that the (1, 3) array is both C and F contiguous, and it is trying to return as large of a 1D Fcontiguous array as it can. However, I didn’t see any way to really force it to go by columns. My best guess was the *itershape* argument though I couldn’t figure out how to get that to work and didn’t see much in the documentation.
Thanks in advance for the help!
 Will
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
On Tue, 20200519 at 17:11 +0100, Eric Wieser wrote:
Hi Will,
To force an iteration to run along certain axes, I believe you should be using `op_axes`. Your diagnosis is correct that `external_loop` is trying to help you be more optimal, since it's purpose is exactly that: optimization.
Unfortunately, if you use `op_axes` you'll run into https://github.com/numpy/numpy/issues/9808.
Yeah, I do not think what you want is possible with nditer. `op_axes` allows you to ignore certain axis, but as Eric points out, that actually ignores that axis entirely. So it acts similar to slicing the whole axis away. I previously thought it could be nice to have a new `arr.iteraxis(axis=None)` command. As a base, it would act like a flat iter, otherwise iterate the axis listed. Maybe with an additional flag whether its allowed to optimize iteration order (although from the python side I expect we do not want to do that by default). The reason is also that I am not sure I like `arr.flat` and `for subarr in arr` too much, because the listoflist like iteration seems only seminatural for an ND array.  Sebastian
Eric
On Tue, 19 May 2020 at 00:42, William Ayd <william.ayd@icloud.com> wrote:
I am trying to use the nditer to traverse each column of a 2D array, returning the column as a 1D array. Consulting the docs, I found this example which works perfectly fine:
In [*65*]: a = np.arange(6).reshape(2,3)
In [*66*]: *for* x *in* np.nditer(a, flags=['external_loop'], order='F'): ...: print(x, end=' ') ...:
[0 3] [1 4] [2 5]
When changing the shape of the input array to (1, 3) however, this doesn’t yield what I am hoping for any more (essentially [0], [1] [2]):
In [*68*]: *for* x *in* np.nditer(a, flags=['external_loop'], order='F'): ...: print(x, end=' ') ...:
[0 1 2]
I suspect this may have to do with the fact that the (1, 3) array is both C and F contiguous, and it is trying to return as large of a 1D Fcontiguous array as it can. However, I didn’t see any way to really force it to go by columns. My best guess was the *itershape* argument though I couldn’t figure out how to get that to work and didn’t see much in the documentation.
Thanks in advance for the help!
 Will
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
participants (4)

Eric Wieser

Hameer Abbasi

Sebastian Berg

William Ayd