On 9/11/07, Robert Kern wrote:

Mike Ressler wrote:

...

The following seems to be a wart: is it expected?

Set up a 10x10 array and some indexing arrays:

a=arange(100) a.shape=(10,10) q=array([0,2,4,6,8]) r=array([0,5])

Suppose I want to extract only the "even" numbered rows from a - then

print a[q,:]

Every fifth column:

print a[:,r]

Only the even rows of every fifth column:

print a[q,r]

---------------------------------------------------------------------------

...

Traceback (most recent call last)

/.../.../.../<ipython console> in <module>()

: shape mismatch: objects cannot be broadcast to a single shape

But, this works:

print a[q,:][:,r]

[[ 0 5] [20 25] [40 45] [60 65] [80 85]]

So why does the a[q,r] form have problems? Thanks for your insights.

It is intended that the form a[q,r] be the general case: q and r are broadcasted against each other to a single shape. The result of the indexing is an array of that broadcasted shape with elements found by using each pair of elements in the broadcasted q and r arrays as indices.

There are operations you can express with this form that you couldn't if the behavior that you expected were the case whereas you can get the result you want relatively straightforwardly.

In [6]: a[q[:,newaxis], r] Out[6]: array([[ 0, 5], [20, 25], [40, 45], [60, 65], [80, 85]])

At the risk of making Robert grumpy: while it is true the form we ended up with is more general I've come to the conclusion that it was a bit of a mistake. In the spirit of making simple things simple and complex things possible, I suspect that having fancy-indexing do the obvious thing here[1] and delegating the more powerful but also more difficult to understand case to a function or method would have been overall more useful. Cases where the multidimensional features of fancy-indexing get used are messy enough that they don't benefit much from the conciseness of the indexing notation, at least in my experience. -- . __ . |-\ . . tim.hochberg@ieee.org [1] Just in case the 'obvious' thing isn't all that obvious: I mean restrict index-arrays to one dimension and have them simply select the given values along the axis. Hmmm. Without giving examples, which I have not time for right now, that's probably not any clearer than saying nothing. Ah well.

Am I right to assume that there is no way elegant way to interact with slices. i.e. Is there anyway to get a[ix_([2,3,6],:,[3,2])] to work? So that the dimension is completely specified? Or perhaps the only way to do this is via a[ix_([2,3,6],range(a.shape[1]),[3,2])] If anyone knows a better way? Thanks, Jonathan. On Tue, Sep 11, 2007 at 6:13 PM, Travis E. Oliphant wrote:

Timothy Hochberg wrote:

...

On 9/11/07, *Robert Kern* mailto:robert.kern@gmail.com> wrote:

    Mike Ressler wrote:     > The following seems to be a wart: is it expected?     >     > Set up a 10x10 array and some indexing arrays:     >     > a=arange(100)     > a.shape=(10,10)     > q=array([0,2,4,6,8])     > r=array([0,5])     >     > Suppose I want to extract only the "even" numbered rows from a -     then     >     > print a[q,:]     >     >     >     > Every fifth column:     >     > print a[:,r]     >     >     >     > Only the even rows of every fifth column:     >     > print a[q,r]     >     >     ---------------------------------------------------------------------------

    >            Traceback (most recent     call last)     >     > /.../.../.../<ipython console> in <module>()     >     > : shape mismatch: objects cannot be     > broadcast to a single shape     >     > But, this works:     >     > print a[q,:][:,r]     >     > [[ 0  5]     >  [20 25]     >  [40 45]     >  [60 65]     >  [80 85]]     >     > So why does the a[q,r] form have problems? Thanks for your insights.

    It is intended that the form a[q,r] be the general case: q and r     are broadcasted     against each other to a single shape. The result of the indexing     is an array of     that broadcasted shape with elements found by using each pair of     elements in the     broadcasted q and r arrays as indices.

    There are operations you can express with this form that you     couldn't if the     behavior that you expected were the case whereas you can get the     result you want     relatively straightforwardly.

    In [6]: a[q[:,newaxis], r]     Out[6]:     array([[ 0,  5],            [20, 25],            [40, 45],            [60, 65],            [80, 85]])

At the risk of making Robert grumpy: while it is true the form we ended up with is more general I've come to the conclusion that it was a bit of a mistake. In the spirit of making simple things simple and complex things possible, I suspect that having fancy-indexing do the obvious thing here[1] and delegating the more powerful but also more difficult to understand case to a function or method would have been overall more useful. Cases where the multidimensional features of fancy-indexing get used are messy enough that they don't benefit much from the conciseness of the indexing notation, at least in my experience.

This is a reasonable argument.    It is reasonable enough that I intentionally made an ix_ function to do what you want.

a[ix_(q,r)]

does as originally expected if a bit more line-noise.

-Travis

_______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion

On 27-Feb-09, at 3:35 PM, David Warde-Farley wrote:

a[[2,3,6],:,:][:,:,[3,2]] should do what you want.

Slightly more elegantly (I always forget about this syntax): a[[2,3,6], ...][..., [3,2]] David

On 28-Feb-09, at 12:27 PM, Jonathan Taylor wrote:

This does seem like the only way to write this nicely. Unfortunately, I think this may be wasteful memory wise (in contrast to what the obvious matlab code would do) as it constructs an array with the whole first index intact at first.

True enough, though if I understand correctly, this is only a _view_ onto the original array, and nothing is immediately copied. So it does waste memory creating a view and then a view on the view, but I don't think it's proportional to the size of the returned array. Maybe Robert or someone else can confirm this. David

On 2-Mar-09, at 12:25 PM, Robert Kern wrote:

a[[2,3,6], ...][..., [3,2]]

You're doing fancy indexing, so there are copies both times.

D'oh! So I guess the only way to avoid the second copy is to do what Jon initially suggested, i.e. a[ix_([2,3,6],range(a.shape[1]),[3,2])] ? I suppose xrange would be better than arange() or range() as it wouldn't create and destroy the list all at once. D

On Thu, Feb 26, 2009 at 21:00, Jonathan Taylor wrote:

Am I right to assume that there is no way elegant way to interact with slices.  i.e. Is there anyway to get

a[ix_([2,3,6],:,[3,2])]

to work?  So that the dimension is completely specified?  Or perhaps the only way to do this is via

a[ix_([2,3,6],range(a.shape[1]),[3,2])]

If anyone knows a better way?

One could probably make ix_() take slice objects, too, to generate the correct arange() in the appropriate place. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

On Tue, Mar 3, 2009 at 03:11, Stéfan van der Walt wrote:

Hi Robert

2009/2/27 Robert Kern :

...

...

a[ix_([2,3,6],range(a.shape[1]),[3,2])]

If anyone knows a better way?

One could probably make ix_() take slice objects, too, to generate the correct arange() in the appropriate place.

I was wondering how one would implement this, since the ix_ function has no knowledge of the dimensions of "a".

No, you're right. It doesn't work. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

Thanks, Robert, for the quick response. On 9/11/07, Robert Kern wrote:

There are operations you can express with this form that you couldn't if the behavior that you expected were the case whereas you can get the result you want relatively straightforwardly.

In [6]: a[q[:,newaxis], r]

Ah, yes, of course. I forgot my broadcasting rules. However, I also thank Timothy Hochberg for the moral support of agreeing that the obvious way (a[q,r]) should be reasonably expected to work. Thanks again for your help - the routine is now working nicely. Mike -- mike.ressler@alum.mit.edu