Reductions with nditer working only with the last axis
Hello, I'm trying to understand how to work with nditer to do a reduction, in my case converting a 3d array into a 2d array. I followed the help here http://docs.scipy.org/doc/numpy/reference/arrays.nditer.html and managed to create a function that applies reduction over the last axis of the input. With this function def nditer_sum(data, red_axes): it = numpy.nditer([data, None], flags=['reduce_ok', 'external_loop'], op_flags=[['readonly'], ['readwrite', 'allocate']], op_axes=[None, red_axes]) it.operands[1][...] = 0 for x, y in it: y[...] = x.sum() return it.operands[1] I can get something equivalent to data.sum(axis=2)
data = numpy.arange(2*3*4).reshape((2,3,4)) nditer_sum(data, [0, 1, -1]) [[ 6 22 38] [54 70 86]] data.sum(axis=2) [[ 6 22 38] [54 70 86]]
So to get something equivalent to data.sum(axis=0) I though that it was enough to change the argument red_axes to [-1, 0,1] But the result is quite different.
data = numpy.arange(2*3*4).reshape((2,3,4)) data.sum(axis=0) [[12 14 16 18] [20 22 24 26] [28 30 32 34]] nditer_sum(data, [-1, 0, 1]) [[210 210 210 210] [210 210 210 210] [210 210 210 210]]
In the for loop inside nditer_sum (for x,y in it:), the iterator is looping 2 times and giving an array of length 12 each time, instead of looping 12 times and giving an array of length 2 each time. I have read the numpy documentation several times and googled about this to no avail. Does anybody have an example of a reduction in the first axis of an array using nditer? Is this a bug? Regards, Sergio
On Thu, Sep 27, 2012 at 6:08 PM, Sergio Pascual <sergio.pasra@gmail.com> wrote:
Hello, I'm trying to understand how to work with nditer to do a reduction, in my case converting a 3d array into a 2d array.
I followed the help here http://docs.scipy.org/doc/numpy/reference/arrays.nditer.html and managed to create a function that applies reduction over the last axis of the input. With this function
def nditer_sum(data, red_axes): it = numpy.nditer([data, None], flags=['reduce_ok', 'external_loop'], op_flags=[['readonly'], ['readwrite', 'allocate']], op_axes=[None, red_axes]) it.operands[1][...] = 0
for x, y in it: y[...] = x.sum()
return it.operands[1]
I can get something equivalent to data.sum(axis=2)
data = numpy.arange(2*3*4).reshape((2,3,4)) nditer_sum(data, [0, 1, -1]) [[ 6 22 38] [54 70 86]] data.sum(axis=2) [[ 6 22 38] [54 70 86]]
So to get something equivalent to data.sum(axis=0) I though that it was enough to change the argument red_axes to [-1, 0,1] But the result is quite different.
data = numpy.arange(2*3*4).reshape((2,3,4)) data.sum(axis=0) [[12 14 16 18] [20 22 24 26] [28 30 32 34]] nditer_sum(data, [-1, 0, 1]) [[210 210 210 210] [210 210 210 210] [210 210 210 210]]
In the for loop inside nditer_sum (for x,y in it:), the iterator is looping 2 times and giving an array of length 12 each time, instead of looping 12 times and giving an array of length 2 each time. I have read the numpy documentation several times and googled about this to no avail.
Does anybody have an example of a reduction in the first axis of an array using nditer? Is this a bug?
Regards, Sergio _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
The example from the link it shows how to do a reduction with y[...] += x. If you would replace x.sum() by += x, then it works.
nditer_sum(data, [-1,0,1]) array([[12, 14, 16, 18], [20, 22, 24, 26], [28, 30, 32, 34]]) data.sum(axis=0) array([[12, 14, 16, 18], [20, 22, 24, 26], [28, 30, 32, 34]])
nditer_sum(data, [0,-1,1]) array([[12, 15, 18, 21], [48, 51, 54, 57]]) data.sum(axis=1) array([[12, 15, 18, 21], [48, 51, 54, 57]])
I think that is because sum() already reduces all axis by default. Regards, Han
Perhaps sum wasn't the best function for this example. I'm going to rework the code with other function Consider a function that operates on an array and returns a number def myfunc(data): return data.min() + 2 * data.max() The function with nditer is: def nditer_fun(data, axes): it = numpy.nditer([data, None], flags=['reduce_ok', 'external_loop'], op_flags=[['readonly'], ['readwrite', 'allocate']], op_axes=[None, axes]) it.operands[1][...] = 0 for x, y in it: y[...] = myfun(x) return it.operands[1] We can compara the result of this function with the result of numpy.apply_along_axis With the data data = numpy.arange(2*3*4).reshape((2,3,4)) the values obtained with axis 0 ara numpy.apply_along_axis(myfun, 0 ,data) [[24 27 30 33] [36 39 42 45] [48 51 54 57]] nditer_fun(data, [-1, 0, 1]) [[58 58 58 58] [58 58 58 58] [58 58 58 58]] Only along the last axis both functions give the same result 2012/10/1 Han Genuit <hangenuit@gmail.com>:
On Thu, Sep 27, 2012 at 6:08 PM, Sergio Pascual <sergio.pasra@gmail.com> wrote:
Hello, I'm trying to understand how to work with nditer to do a reduction, in my case converting a 3d array into a 2d array.
I followed the help here http://docs.scipy.org/doc/numpy/reference/arrays.nditer.html and managed to create a function that applies reduction over the last axis of the input. With this function
def nditer_sum(data, red_axes): it = numpy.nditer([data, None], flags=['reduce_ok', 'external_loop'], op_flags=[['readonly'], ['readwrite', 'allocate']], op_axes=[None, red_axes]) it.operands[1][...] = 0
for x, y in it: y[...] = x.sum()
return it.operands[1]
I can get something equivalent to data.sum(axis=2)
data = numpy.arange(2*3*4).reshape((2,3,4)) nditer_sum(data, [0, 1, -1]) [[ 6 22 38] [54 70 86]] data.sum(axis=2) [[ 6 22 38] [54 70 86]]
So to get something equivalent to data.sum(axis=0) I though that it was enough to change the argument red_axes to [-1, 0,1] But the result is quite different.
data = numpy.arange(2*3*4).reshape((2,3,4)) data.sum(axis=0) [[12 14 16 18] [20 22 24 26] [28 30 32 34]] nditer_sum(data, [-1, 0, 1]) [[210 210 210 210] [210 210 210 210] [210 210 210 210]]
In the for loop inside nditer_sum (for x,y in it:), the iterator is looping 2 times and giving an array of length 12 each time, instead of looping 12 times and giving an array of length 2 each time. I have read the numpy documentation several times and googled about this to no avail.
Does anybody have an example of a reduction in the first axis of an array using nditer? Is this a bug?
Regards, Sergio _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
The example from the link it shows how to do a reduction with y[...] += x. If you would replace x.sum() by += x, then it works.
nditer_sum(data, [-1,0,1]) array([[12, 14, 16, 18], [20, 22, 24, 26], [28, 30, 32, 34]]) data.sum(axis=0) array([[12, 14, 16, 18], [20, 22, 24, 26], [28, 30, 32, 34]])
nditer_sum(data, [0,-1,1]) array([[12, 15, 18, 21], [48, 51, 54, 57]]) data.sum(axis=1) array([[12, 15, 18, 21], [48, 51, 54, 57]])
I think that is because sum() already reduces all axis by default.
Regards, Han _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
-- Sergio Pascual http://guaix.fis.ucm.es/~spr +34 91 394 5018 gpg fingerprint: 5203 B42D 86A0 5649 410A F4AC A35F D465 F263 BCCC Departamento de Astrofísica -- Universidad Complutense de Madrid (Spain)
participants (3)
-
Han Genuit -
Sergio Pascual -
Sergio Pascual