shoehorn c-structured data into Numpy

[Scott David Daniels]

MRAB wrote:
> Helmut Fritz wrote:
>> I have binary output 
>> from a Fortran program that is in a big-endian C-structured binary 
>> file.  The output can be very variable and many options create 
>> different orderings in the binary file. So I'd like to keep the 
>> header-reading in python.
>>
>> Anyhoo, I've so far been able to read the output with the struct 
>> module.  But my question is how do I create numpy arrays from the bits 
>> of the file I want?
 >> TC1 = np.frombuffer(struct.unpack(">%df" % ncells,
 >>                                data.read(4*ncells))[0], dtype=float)

Note both struct and numpy.frombuffer unpack data from a buffer.
Do it only once:

If you are going to a numpy array:
     TC1 = np.frombuffer(data.read(4*ncells)), dtype=float)
     # and possibly:
     TC1.byteswap(True) # True to byteswap in place.

Or (even better -- less copying):

     TC1 = np.fromfile(data, count=ncells, dtype=float)
     # and again, possibly:
     TC1.byteswap(True) # True to byteswap in place.

--Scott David Daniels
Scott.Daniels at Acm.Org