[Numpy-discussion] Using np.frombuffer and cffi.buffer on array of C structs (problem with struct member padding)

[Chris Barker - NOAA Federal]

Sent from my iPhone

> On Jan 25, 2018, at 6:51 AM, Joe <solarjoe at posteo.org> wrote:
>
> Hello,
>
> how I could dynamically handle the dtype of a structured array when reading
> an array of C structs with np.frombuffer (regarding the member padding in the struct).
>
> So far I manually adjusted the dtype of the structured array and added a field for the padding,
> this works on a small scale.
> The structs are not defined within my code, but within a third party and
> basically I am looking for no-worry hassle free way to handle this, because there are a lot of structs
>
> Is there some smart way to do this in Numpy?

The numpy dtype constructor takes an “align” keyword that will pad it for you.

However, if these strict are coming from a lib compiled by a third
party, I’m not sure you can count on the alignment rules being the
same.

So maybe you will need to use the cffi functions :-(

-CHB


>
> So far the best approach seems to parse the struct with the cffi functions
> ffi.sizeof(), ffi.offsetof() and maybe ffi.alignof() to find out where the padding
> happens and add it dynamically to the dtype. But maybe someone has a
> smarter idea how to solve this.
>
> You can find a more detailed description and a working example here:
>
> https://stackoverflow.com/questions/48423725/how-to-handle-member-padding-in-struct-when-reading-cffi-buffer-with-numpy-fromb
>
> Kind regards,
> Joe
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion