Struct on on x86_64 mac os x

[Mark Tolonen]

"Tommy Grav" <tgrav at pha.jhu.edu> wrote in message 
news:D705AB12-0BEE-495A-B1E5-C43245E403C8 at pha.jhu.edu...
>I have created a binary file that saves this struct from some C code:
>
>   struct recOneData {
>          char label[3][84];
>          char constName[400][6];
>        double timeData[3];
>      long int numConst;
>        double AU;
>        double EMRAT;
>      long int coeffPtr[12][3];
>      long int DENUM;
>      long int libratPtr[3];
>      };
>
> I try to read this file with python (ActiveState 2.6.3 on x86_64 Mac  OS X 
> 10.6.1) using the
> code below the hdrData and constNData are fine, while from the  timeData 
> onwards there
> are obvious problems. The code below works fine for a 32bit binary  read 
> with i386 python
> 2.5.2. Does anyone know what the proper format of a C double and long  int 
> is for a x86_64
> binary?
>
>     def read_header(cls):
>         hdrData = "84s"*3
>         constNData = "6s"*400
>         timeData = "d"*3
>         numData = "ldd"
>         coeffData = "3l"*12
>         denumData = "l"
>         libPtrData = "lll"
>         constData = "400d"
>         hformat = hdrData + constNData + timeData + numData +  coeffData + 
> denumData + libPtrData
>         header = struct.unpack(hdrData,cls.JPLeph.read(struct.calcsize 
> (hdrData)))
>
>         constN = struct.unpack(constNData,cls.JPLeph.read 
> (struct.calcsize(constNData)))
>
>         # Reading in the time data
>         cls.tstart,cls.tend,cls.tstep = struct.unpack 
> (timeData,cls.JPLeph.read(struct.calcsize(timeData)))
>
>         # Read in the number of constants and the values for AU and  emrat
>         nconst, cls.au, cls.emrat = struct.unpack 
> (numData,cls.JPLeph.read(struct.calcsize(numData)))
>
>         # Reading in the coefficient pointers
>         cls.coeff = struct.unpack(coeffData,cls.JPLeph.read 
> (struct.calcsize(coeffData)))
>         # Read in the DENUM variable and the libratPtr(3)
>         (cls.denum, ) = struct.unpack(denumData,cls.JPLeph.read 
> (struct.calcsize(denumData)))
>         cls.libPtr = struct.unpack(libPtrData,cls.JPLeph.read 
> (struct.calcsize(libPtrData)))
>
>         if cls.denum == 405:
>             cls.asize = 1018
>         elif cls.denum == 200:
>             cls.asize = 826
>         else:
>             raise ValueError("JPL ephem file is in unknown format %d"  % 
> (cls.denum))

Try unpacking in one big chunk using 'hformat' above.  struct should do the 
right thing with structure unpacking if it knows how the entire structure is 
laid out.

C structures use padding to align data elements on "natural" boundaries 
because it is more efficient for a processor to read DWORDs on DWORD 
(4-byte) boundaries, WORDs on 2-byte boundaries, etc.  The struct module is 
aware of this.  Below a long is 4 bytes and a double is 8 bytes.  If the 
long is first the double would be at offset 4 (not an 8-byte boundary) so 4 
bytes of padding are inserted.  If the double is first the long is at offset 
8 (a 4-byte boundary) and the padding isn't inserted.

>>> struct.calcsize('ld')
16
>>> struct.calcsize('dl')
12

In your case above, the size of the char arrays before the first double is 
not a multiple of 8, so padding is inserted.  Since your example read the 
chunks separately, struct didn't know to insert the padding.

-Mark