Conditional update of recarray field
Hi, I try to update values in a single field of numpy record array based on a condition defined in another array. I found that that the result depends on the order in which I apply the boolean indices/field names. For example: cond = np.zeros(5, dtype=np.bool) cond[2:] = True X = np.rec.fromarrays([np.arange(5)], names='a') X[cond]['a'] = -1 print X returns: [(0,) (1,) (2,) (3,) (4,)] (the values were not updated) X['a'][cond] = -1 print X returns: [(0,) (1,) (-1,) (-1,) (-1,)] (it worked this time). I find this behaviour very confusing. Is it expected? Would it be possible to emit a warning message in the case of "faulty" assignments? Bartosz
On 11/28/12 1:47 PM, Bartosz wrote:
Hi,
I try to update values in a single field of numpy record array based on a condition defined in another array. I found that that the result depends on the order in which I apply the boolean indices/field names.
For example:
cond = np.zeros(5, dtype=np.bool) cond[2:] = True X = np.rec.fromarrays([np.arange(5)], names='a') X[cond]['a'] = -1 print X
returns: [(0,) (1,) (2,) (3,) (4,)] (the values were not updated)
X['a'][cond] = -1 print X
returns: [(0,) (1,) (-1,) (-1,) (-1,)] (it worked this time).
I find this behaviour very confusing. Is it expected?
Yes, it is. In the first idiom, X[cond] is a fancy indexing operation and the result is not a view, so what you are doing is basically modifying the temporary object that results from the indexing. In the second idiom, X['a'] is returning a *view* of the original object, so this is why it works.
Would it be possible to emit a warning message in the case of "faulty" assignments?
The only solution that I can see for this is that the fancy indexing would return a view, and not a different object, but NumPy containers are not prepared for this. -- Francesc Alted
Thanks for answer, Francesc. I understand now that fancy indexing returns a copy of a recarray. Is it also true for standard ndarrays? If so, I do not understand why X['a'][cond]=-1 should work. Cheers, Bartosz On Wed 28 Nov 2012 03:05:37 PM CET, Francesc Alted wrote:
On 11/28/12 1:47 PM, Bartosz wrote:
Hi,
I try to update values in a single field of numpy record array based on a condition defined in another array. I found that that the result depends on the order in which I apply the boolean indices/field names.
For example:
cond = np.zeros(5, dtype=np.bool) cond[2:] = True X = np.rec.fromarrays([np.arange(5)], names='a') X[cond]['a'] = -1 print X
returns: [(0,) (1,) (2,) (3,) (4,)] (the values were not updated)
X['a'][cond] = -1 print X
returns: [(0,) (1,) (-1,) (-1,) (-1,)] (it worked this time).
I find this behaviour very confusing. Is it expected?
Yes, it is. In the first idiom, X[cond] is a fancy indexing operation and the result is not a view, so what you are doing is basically modifying the temporary object that results from the indexing. In the second idiom, X['a'] is returning a *view* of the original object, so this is why it works.
Would it be possible to emit a warning message in the case of "faulty" assignments?
The only solution that I can see for this is that the fancy indexing would return a view, and not a different object, but NumPy containers are not prepared for this.
Hey Bartosz, On 11/28/12 3:26 PM, Bartosz wrote:
Thanks for answer, Francesc.
I understand now that fancy indexing returns a copy of a recarray. Is it also true for standard ndarrays? If so, I do not understand why X['a'][cond]=-1 should work.
Yes, that's a good question. No, in this case the boolean array `cond` is passed to the __setitem__() of the original view, so this is why this works. The first idiom is concatenating the fancy indexing with another indexing operation, and NumPy needs to create a temporary for executing this, so the second indexing operation acts over a copy, not a view. And yes, fancy indexing returning a copy is standard for all ndarrays. Hope it is clearer now (although admittedly it is a bit strange at first sight), -- Francesc Alted
I got it. Thanks! Now I see why this is non-trivial to fix it. However, it might be also a source of very-hard-to-find bugs. It might be worth discussing this non-intuitive example in the documentation. Cheers, Bartosz
Thanks for answer, Francesc.
I understand now that fancy indexing returns a copy of a recarray. Is it also true for standard ndarrays? If so, I do not understand why X['a'][cond]=-1 should work.
Yes, that's a good question. No, in this case the boolean array `cond` is passed to the __setitem__() of the original view, so this is why this works. The first idiom is concatenating the fancy indexing with another indexing operation, and NumPy needs to create a temporary for executing this, so the second indexing operation acts over a copy, not a view.
And yes, fancy indexing returning a copy is standard for all ndarrays.
Hope it is clearer now (although admittedly it is a bit strange at first sight),
participants (2)
-
Bartosz
-
Francesc Alted