Reductions with nditer working only with the last axis
Hello, I'm trying to understand how to work with nditer to do a reduction, in my case converting a 3d array into a 2d array. I followed the help here http://docs.scipy.org/doc/numpy/reference/arrays.nditer.html and managed to create a function that applies reduction over the last axis of the input. With this function def nditer_sum(data, red_axes): it = numpy.nditer([data, None], flags=['reduce_ok', 'external_loop'], op_flags=[['readonly'], ['readwrite', 'allocate']], op_axes=[None, red_axes]) it.operands[1][...] = 0 for x, y in it: y[...] = x.sum() return it.operands[1] I can get something equivalent to data.sum(axis=2)
data = numpy.arange(2*3*4).reshape((2,3,4)) nditer_sum(data, [0, 1, 1]) [[ 6 22 38] [54 70 86]] data.sum(axis=2) [[ 6 22 38] [54 70 86]]
So to get something equivalent to data.sum(axis=0) I though that it was enough to change the argument red_axes to [1, 0,1] But the result is quite different.
data = numpy.arange(2*3*4).reshape((2,3,4)) data.sum(axis=0) [[12 14 16 18] [20 22 24 26] [28 30 32 34]] nditer_sum(data, [1, 0, 1]) [[210 210 210 210] [210 210 210 210] [210 210 210 210]]
In the for loop inside nditer_sum (for x,y in it:), the iterator is looping 2 times and giving an array of length 12 each time, instead of looping 12 times and giving an array of length 2 each time. I have read the numpy documentation several times and googled about this to no avail. Does anybody have an example of a reduction in the first axis of an array using nditer? Is this a bug? Regards, Sergio
On Thu, Sep 27, 2012 at 6:08 PM, Sergio Pascual
Hello, I'm trying to understand how to work with nditer to do a reduction, in my case converting a 3d array into a 2d array.
I followed the help here http://docs.scipy.org/doc/numpy/reference/arrays.nditer.html and managed to create a function that applies reduction over the last axis of the input. With this function
def nditer_sum(data, red_axes): it = numpy.nditer([data, None], flags=['reduce_ok', 'external_loop'], op_flags=[['readonly'], ['readwrite', 'allocate']], op_axes=[None, red_axes]) it.operands[1][...] = 0
for x, y in it: y[...] = x.sum()
return it.operands[1]
I can get something equivalent to data.sum(axis=2)
data = numpy.arange(2*3*4).reshape((2,3,4)) nditer_sum(data, [0, 1, 1]) [[ 6 22 38] [54 70 86]] data.sum(axis=2) [[ 6 22 38] [54 70 86]]
So to get something equivalent to data.sum(axis=0) I though that it was enough to change the argument red_axes to [1, 0,1] But the result is quite different.
data = numpy.arange(2*3*4).reshape((2,3,4)) data.sum(axis=0) [[12 14 16 18] [20 22 24 26] [28 30 32 34]] nditer_sum(data, [1, 0, 1]) [[210 210 210 210] [210 210 210 210] [210 210 210 210]]
In the for loop inside nditer_sum (for x,y in it:), the iterator is looping 2 times and giving an array of length 12 each time, instead of looping 12 times and giving an array of length 2 each time. I have read the numpy documentation several times and googled about this to no avail.
Does anybody have an example of a reduction in the first axis of an array using nditer? Is this a bug?
Regards, Sergio _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
The example from the link it shows how to do a reduction with y[...] += x. If you would replace x.sum() by += x, then it works.
nditer_sum(data, [1,0,1]) array([[12, 14, 16, 18], [20, 22, 24, 26], [28, 30, 32, 34]]) data.sum(axis=0) array([[12, 14, 16, 18], [20, 22, 24, 26], [28, 30, 32, 34]])
nditer_sum(data, [0,1,1]) array([[12, 15, 18, 21], [48, 51, 54, 57]]) data.sum(axis=1) array([[12, 15, 18, 21], [48, 51, 54, 57]])
I think that is because sum() already reduces all axis by default. Regards, Han
Perhaps sum wasn't the best function for this example. I'm going to
rework the code with other function
Consider a function that operates on an array and returns a number
def myfunc(data):
return data.min() + 2 * data.max()
The function with nditer is:
def nditer_fun(data, axes):
it = numpy.nditer([data, None],
flags=['reduce_ok', 'external_loop'],
op_flags=[['readonly'], ['readwrite', 'allocate']],
op_axes=[None, axes])
it.operands[1][...] = 0
for x, y in it:
y[...] = myfun(x)
return it.operands[1]
We can compara the result of this function with the result of
numpy.apply_along_axis
With the data
data = numpy.arange(2*3*4).reshape((2,3,4))
the values obtained with axis 0 ara
numpy.apply_along_axis(myfun, 0 ,data)
[[24 27 30 33]
[36 39 42 45]
[48 51 54 57]]
nditer_fun(data, [1, 0, 1])
[[58 58 58 58]
[58 58 58 58]
[58 58 58 58]]
Only along the last axis both functions give the same result
2012/10/1 Han Genuit
On Thu, Sep 27, 2012 at 6:08 PM, Sergio Pascual
wrote: Hello, I'm trying to understand how to work with nditer to do a reduction, in my case converting a 3d array into a 2d array.
I followed the help here http://docs.scipy.org/doc/numpy/reference/arrays.nditer.html and managed to create a function that applies reduction over the last axis of the input. With this function
def nditer_sum(data, red_axes): it = numpy.nditer([data, None], flags=['reduce_ok', 'external_loop'], op_flags=[['readonly'], ['readwrite', 'allocate']], op_axes=[None, red_axes]) it.operands[1][...] = 0
for x, y in it: y[...] = x.sum()
return it.operands[1]
I can get something equivalent to data.sum(axis=2)
data = numpy.arange(2*3*4).reshape((2,3,4)) nditer_sum(data, [0, 1, 1]) [[ 6 22 38] [54 70 86]] data.sum(axis=2) [[ 6 22 38] [54 70 86]]
So to get something equivalent to data.sum(axis=0) I though that it was enough to change the argument red_axes to [1, 0,1] But the result is quite different.
data = numpy.arange(2*3*4).reshape((2,3,4)) data.sum(axis=0) [[12 14 16 18] [20 22 24 26] [28 30 32 34]] nditer_sum(data, [1, 0, 1]) [[210 210 210 210] [210 210 210 210] [210 210 210 210]]
In the for loop inside nditer_sum (for x,y in it:), the iterator is looping 2 times and giving an array of length 12 each time, instead of looping 12 times and giving an array of length 2 each time. I have read the numpy documentation several times and googled about this to no avail.
Does anybody have an example of a reduction in the first axis of an array using nditer? Is this a bug?
Regards, Sergio _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
The example from the link it shows how to do a reduction with y[...] += x. If you would replace x.sum() by += x, then it works.
nditer_sum(data, [1,0,1]) array([[12, 14, 16, 18], [20, 22, 24, 26], [28, 30, 32, 34]]) data.sum(axis=0) array([[12, 14, 16, 18], [20, 22, 24, 26], [28, 30, 32, 34]])
nditer_sum(data, [0,1,1]) array([[12, 15, 18, 21], [48, 51, 54, 57]]) data.sum(axis=1) array([[12, 15, 18, 21], [48, 51, 54, 57]])
I think that is because sum() already reduces all axis by default.
Regards, Han _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
 Sergio Pascual http://guaix.fis.ucm.es/~spr +34 91 394 5018 gpg fingerprint: 5203 B42D 86A0 5649 410A F4AC A35F D465 F263 BCCC Departamento de Astrofísica  Universidad Complutense de Madrid (Spain)
participants (3)

Han Genuit

Sergio Pascual

Sergio Pascual