[Numpy-discussion] Question about structure arrays

[Nathaniel Smith]

On Sat, Nov 7, 2015 at 1:18 PM, aerojockey <pythondev1 at aerojockey.com> wrote:
> Hello,
>
> Recently I made some changes to a program I'm working on, and found that the
> changes made it four times slower than before.  After some digging, I found
> out that one of the new costs was that I added structure arrays.  Inside a
> low-level loop, I create a structure array, populate it Python, then turn it
> over to some handwritten C code for processing.  It turned out that, when
> passed a structure array as a dtype, numpy has to parse the dtype, which
> included calls to re.match and eval.
>
> Now, this is not a big deal for me to work around by using ordinary slicing
> and such, and also I can improve things by reusing arrays.  Since this is
> inner loop stuff, sacrificing readability for speed is an appropriate
> tradeoff.
>
> Nevertheless, I was curious if there was a way (or any plans for there to be
> a way) to compile a struture array dtype.  I realize it's not the
> bread-and-butter of numpy, but it turned out to be a very convenient feature
> for my use case (populating an array of structures to pass off to C).

Does it help to turn your dtype string into a dtype object and then
pass the dtype object around? E.g.

In [1]: dt = np.dtype("i4,i4")

In [2]: np.zeros(2, dtype=dt)
Out[2]:
array([(0, 0), (0, 0)],
      dtype=[('f0', '<i4'), ('f1', '<i4')])

-n

-- 
Nathaniel J. Smith -- http://vorpus.org