[Numpy-discussion] Fancy Indexing of Structured Arrays is Slow
dave.hirschfeld at gmail.com
Fri May 16 04:41:48 EDT 2014
Sebastian Berg <sebastian <at> sipsolutions.net> writes:
> On Do, 2014-05-15 at 12:31 +0000, Dave Hirschfeld wrote:
> > As can be seen from the code below (or in the notebook linked beneath)
> > indexing of a structured array is twice as slow as indexing both fields
> > independently - making it 4x slower?
> The non-vanilla types tend to be somewhat more efficient with these
> things and the first indexing does not copy so it is rather fast. I did
> not check the code, but we use (also in the new one for this operation)
> the copyswap function on individual elements (only for non-trivial
> copies in 1.9 in later, making the difference even larger), and this is
> probably not specialized to the specific void type so it probably has to
> do call the copyswap for every field (and first get the fields). All
> that work would be done for every element.
> If you are interested in this, you could check the fancy indexing inner
> loop and see if replacing the copyswap with the specialized strided
> transfer functions (it is used further down in a different branch of the
> loop) actually makes things faster. I would expect so for some void
> types anyway, but not sure in general.
> - Sebastian
Thanks for the explanation and pointers - it sounds like a good opportunity
for getting stuck into the internals of numpy which I've been meaning to do.
I'm not sure I've got the required skills but I'm sure it will be a good
Unfortunately it won't likely be in the immediate future that'll I'll have
the time to do so. In the meantime I can live with indexing the fields
More information about the NumPy-Discussion