...

I apologise if I'm asking an obvious question or one that has already been addressed.

I've tried to understand the documentation in the numpy manual on slicing, but I'm a bit lost. I'm trying to do indexing using both slices and index lists. I have a problem when I do something like:

x[0, :, [0,1,2]]

Here are a couple of examples:

...

...

...

a = numpy.arange(6).reshape(2,3) print a [[0 1 2] [3 4 5]] print a[:, [0,1,2]] # example 1 - this works as I expected [[0 1 2] [3 4 5]] b = numpy.arange(6).reshape(1,2,3) print b [[[0 1 2] [3 4 5]]] print b[0, :, [0,1,2]] # example 2 - this seems to be the transpose of what I was expecting [[0 3] [1 4] [2 5]] print b[0, [[0],[1]], [[0,1,2]]] # example 3 - this is what I expected [[0 1 2] [3 4 5]]

Am I doing something wrong? Why do we get different behaviour in example 2 compared with example 1 or example 3?

(I'm using numpy 1.0.3.1 on python 2.4.1 for windows, but I've tried some more recent versions of numpy as well.)

mattp

that's how it works, whether we like it or not.

see thread with title "is it a bug?" starting march 11

Josef

Thanks Josef, I've looked over "is it a bug" thread, and realise that it is very relevant! But I'm still lost. Robert Kern wrote: "It's certainly weird, but it's working as designed. Fancy indexing via arrays is a separate subsystem from indexing via slices. Basically, fancy indexing decides the outermost shape of the result (e.g. the leftmost items in the shape tuple). If there are any sliced axes, they are *appended* to the end of that shape tuple." I see that's the case in example 2, but not in example 1 (above). Josef, I also see your example doesn't fit this explanation:

...

...

x = np.arange(30).reshape(3,5,2) idx = np.array([0,1]); e = x[:,[0,1],0]; e.shape (3, 2) idx = np.array([0,1]); e = x[:,:2,0]; e.shape (3, 2)

Travis Oliphant wrote: Referencing my previous post on this topic. In this case, it is unambiguous to replace dimensions 1 and 2 with the result of broadcasting idx and idx together. Thus the (5,6) dimensions is replaced by the (2,) result of indexing leaving the outer dimensions in-tact, thus (4,2,7) is the result. I'm unclear on when something is regarded as "unambiguous"; I don't really get how the rules work. I'm trying to build something where I can do (for "a" having a shape (n1,n2,n3,...)): a[i1, i2, i3, ...] where i1, i2, i3 can be * a single index: eg a[3] * a slice: eg a[:3] * a list of keys: eg a[[1,2,3]] and the interpretation of this should yield: * no corresponding dimension if a single index is used * a dimension of length of the slice if a slice is used * a dimension of length of the list if a list is used I currently apply the following logic: * look through the index coordinates that are being applied * if there are multiple list-of-key indices, then reshape them so that they will broadcast to agree: a[[1,2,3], [4,5]] --> a[[[1],[2],[3]], [[4,5]]] * note if there are any slices. If so, I assume (as per Robert Kern's remark) that the dimensions corresponding to the slices are going to be appended to the end. So I make sure that I transpose my result at the end to correct for this. When I do all this, I get example 2 behaving like example 3, but example 1 then doesn't work. I'm not trying to get the discussion list to do my work for me, but I'm pretty confused as to when dimensions get swapped and when they don't; when something is "ambiguous" and when it is "unambiguous". Any help appreciated, thanks, matt -- This message and any attachments are confidential, proprietary, and may be privileged. If this message was misdirected, Barclays Global Investors (BGI) does not waive any confidentiality or privilege. If you are not the intended recipient, please notify us immediately and destroy the message without disclosing its contents to anyone. Any distribution, use or copying of this e-mail or the information it contains by other than an intended recipient is unauthorized. The views and opinions expressed in this e-mail message are the author's own and may not reflect the views and opinions of BGI, unless the author is authorized by BGI to express such views or opinions on its behalf. All email sent to or from this address is subject to electronic storage and review by BGI. Although BGI operates anti-virus programs, it does not accept responsibility for any damage whatsoever caused by viruses being passed.

Sorry group. I found Travis Oliphant's earlier 12 March post (that didn't show up in the same thread), and found the answer to my question. matt

...

...

I apologise if I'm asking an obvious question or one that has already been addressed.

I've tried to understand the documentation in the numpy manual on slicing, but I'm a bit lost. I'm trying to do indexing using both slices and index lists. I have a problem when I do something like:

x[0, :, [0,1,2]]

Here are a couple of examples:

...

...

...

a = numpy.arange(6).reshape(2,3) print a [[0 1 2] [3 4 5]] print a[:, [0,1,2]] # example 1 - this works as I expected [[0 1 2] [3 4 5]] b = numpy.arange(6).reshape(1,2,3) print b [[[0 1 2] [3 4 5]]] print b[0, :, [0,1,2]] # example 2 - this seems to be the transpose of what I was expecting [[0 3] [1 4] [2 5]] print b[0, [[0],[1]], [[0,1,2]]] # example 3 - this is what I expected [[0 1 2] [3 4 5]]

Am I doing something wrong? Why do we get different behaviour in example 2 compared with example 1 or example 3?

(I'm using numpy 1.0.3.1 on python 2.4.1 for windows, but I've tried some more recent versions of numpy as well.)

mattp

that's how it works, whether we like it or not.

see thread with title "is it a bug?" starting march 11

Josef

Thanks Josef,

I've looked over "is it a bug" thread, and realise that it is very relevant! But I'm still lost. Robert Kern wrote:

"It's certainly weird, but it's working as designed. Fancy indexing via arrays is a separate subsystem from indexing via slices. Basically, fancy indexing decides the outermost shape of the result (e.g. the leftmost items in the shape tuple). If there are any sliced axes, they are *appended* to the end of that shape tuple."

I see that's the case in example 2, but not in example 1 (above). Josef, I also see your example doesn't fit this explanation:

...

...

...

x = np.arange(30).reshape(3,5,2) idx = np.array([0,1]); e = x[:,[0,1],0]; e.shape (3, 2) idx = np.array([0,1]); e = x[:,:2,0]; e.shape (3, 2)

Travis Oliphant wrote:

Referencing my previous post on this topic. In this case, it is unambiguous to replace dimensions 1 and 2 with the result of broadcasting idx and idx together. Thus the (5,6) dimensions is replaced by the (2,) result of indexing leaving the outer dimensions in-tact, thus (4,2,7) is the result.

I'm unclear on when something is regarded as "unambiguous"; I don't really get how the rules work.

I'm trying to build something where I can do (for "a" having a shape (n1,n2,n3,...)):

a[i1, i2, i3, ...]

where i1, i2, i3 can be * a single index: eg a[3] * a slice: eg a[:3] * a list of keys: eg a[[1,2,3]] and the interpretation of this should yield: * no corresponding dimension if a single index is used * a dimension of length of the slice if a slice is used * a dimension of length of the list if a list is used

I currently apply the following logic: * look through the index coordinates that are being applied * if there are multiple list-of-key indices, then reshape them so that they will broadcast to agree: a[[1,2,3], [4,5]] --> a[[[1],[2],[3]], [[4,5]]] * note if there are any slices. If so, I assume (as per Robert Kern's remark) that the dimensions corresponding to the slices are going to be appended to the end. So I make sure that I transpose my result at the end to correct for this.

When I do all this, I get example 2 behaving like example 3, but example 1 then doesn't work. I'm not trying to get the discussion list to do my work for me, but I'm pretty confused as to when dimensions get swapped and when they don't; when something is "ambiguous" and when it is "unambiguous".

Any help appreciated, thanks, matt

-- This message and any attachments are confidential, proprietary, and may be privileged. If this message was misdirected, Barclays Global Investors (BGI) does not waive any confidentiality or privilege. If you are not the intended recipient, please notify us immediately and destroy the message without disclosing its contents to anyone. Any distribution, use or copying of this e-mail or the information it contains by other than an intended recipient is unauthorized. The views and opinions expressed in this e-mail message are the author's own and may not reflect the views and opinions of BGI, unless the author is authorized by BGI to express such views or opinions on its behalf. All email sent to or from this address is subject to electronic storage and review by BGI. Although BGI operates anti-virus programs, it does not accept responsibility for any damage whatsoever caused by viruses being passed.