[Numpy-discussion] problem with FortranFile

[Neil Martinsen-Burrell]

On 2009-11-08 17:33 , Brennan Williams wrote:
> I'm using FortranFile to read a binary Fortran file.
> It has a bit of header data at the top of the file which I'm reading
> with a combination of readString and struct.unpack
> This is then followed by a number of lines/records, each of which has 20
> double precision reals/floats.
> For some reason it reads the first 19 ok and then things start going wrong.
>
> To read the double precision data I'm using....
>
>            for il in range(0,nlines):
>                try:
>                    darray=f.readReals('d')
>               except:
>                   print 'problem reading well data line',il
>
>
> I've added print statements to my code and to fortranfile.py and the
> print output I get is....
>
> line 19
> readRecord:l= 160
> readRecord:len(data_str)= 160
> readRecord:check_size= 160
> readReals:len(data_str)= 160
> calcsizeprec= 8
> num= 20
> numbers
> (4.2843303680419922, 0.0, 0.0, 0.0, 4955.73974609375, 0.0, -1000.0,
> -1000.0, 0.0, 0.0, 0.0, 21.22749137878418, 0.0, 0.0, 0.0, 0.0,
> 0.2054678201675415, 0.0, 6386.78271484375, 6356.27001953125)
> il= 19 dsize= 20
> darray= [  4.28433037e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00
>     4.95573975e+03   0.00000000e+00  -1.00000000e+03  -1.00000000e+03
>     0.00000000e+00   0.00000000e+00   0.00000000e+00   2.12274914e+01
>     0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00
>     2.05467820e-01   0.00000000e+00   6.38678271e+03   6.35627002e+03]
> ---------------------
> line 20
> readRecord:l= 160
> readRecord:len(data_str)= 132
> problem reading well data line 20
>
>
> line 19 is ok, readReals calls readRecord and the length of the data is
> 160 and check_size = l etc etc.
>
> line 20 is not ok, the 4-byte length value at the start of the record is
> 160 but the data_str=self.read(l) line only gets 132 bytes and not 160.
>
> The data file itself is ok as I've written a small Fortran program to
> read it.
>
> So, am I doing something wrong?
>
> I have a little test .py file and data file I can include if it helps.

I've run into this problem before, but I don't recall what I did to work 
around it.  The problem is this: python's f.read(SIZE) method returns at 
most SIZE bytes.  There is no guarantee that it will actually return 
that number of bytes.

The right fix is to call self.read in a loop until you get exactly the 
right number of bytes.  The attached version of fortranfile.py should do 
the trick.  Let me know if it does or doesn't help.

[As an aside, fortranfile.py is code that I've written that isn't part 
of Numpy and perhaps the right place for any discussions of it is off-list.]

-Neil
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: fortranfile.py
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20091108/514c57ef/attachment.ksh>