[AstroPy] PyFITS IndexError

[Michael Droettboom]

The problem is related to the handling of CONTINUE cards.

The linked file has a surprising use of CONTINUE cards that pyfits has 
not been tested for.

hsa.fits has the following:

   TTYPE17 = 'LOF_code'           / &
   CONTINUE  '' / &

The CONTINUE card has an empty value (which is what is tripping up 
pyfits).  This is probably not strictly against the specification [1], 
but it is meaningless, and it's not clear why the system that generated 
the file would be spitting those out.  Also, the '&' is supposed to go 
inside the value (i.e. inside the '') not inside the comment part.  In 
any case, the second line above should not contain an &, since it is the 
last line of the continued value.

[1] http://heasarc.gsfc.nasa.gov/docs/heasarc/ofwg/docs/ofwg_recomm/r13.html

The attached patch against PyFITS allows it to handle this file.  
However, I would also recommend reporting the bug to the author(s) of 
the tool that generated this file.

Mike

On 11/16/2010 09:28 AM, Miguel de Val Borro wrote:
> Hello,
>
> I have problems reading FITS files from the Herschel Science Archive
> generated using their Java data processing system. The file
> http://www.mps.mpg.de/data/outgoing/deval/hsa.fits
> has a binary table that cannot be accessed by index or name as shown
> below.
>
> When the line hdulist[1].data.field(0) is run a few times in the
> interpreter the data is finally printed.  How can I read this kind of
> file using PyFITS?
>
> Best regards,
> Miguel
>
>    
>>>> pyfits.__version__
>>>>          
> '2.3.1'
>    
>>>> hdulist = pyfits.open('hsa.fits')
>>>> hdulist[1].data.field(0)
>>>>          
> ---------------------------------------------------------------------------
> IndexError                                Traceback (most recent call last)
>
> /home/miguel/HssO/103P/data/fits/<ipython console>  in<module>()
>
> /usr/local/lib/python2.6/dist-packages/pyfits/core.pyc in __getattr__(self, attr)
>     6240             if size:
>     6241                 self._file.seek(self._datLoc)
> ->  6242                 data = _get_tbdata(self)
>     6243                 data._coldefs = self.columns
>     6244                 data.formats = self.columns.formats
>
> /usr/local/lib/python2.6/dist-packages/pyfits/core.pyc in _get_tbdata(hdu)
>     5165     Get the table data from input (an HDU object).
>     5166     """
> ->  5167     tmp = hdu.columns
>     5168     # get the right shape for the data part of the random group,
>
>     5169     # since binary table does not support ND yet
>
>
> /usr/local/lib/python2.6/dist-packages/pyfits/core.pyc in __getattr__(self, attr)
>     6251             class_name = str(self.__class__)
>     6252             class_name = class_name[class_name.rfind('.')+1:-2]
> ->  6253             self.__dict__[attr] = ColDefs(self, tbtype=class_name)
>     6254
>     6255         elif attr == '_theap':
>
> /usr/local/lib/python2.6/dist-packages/pyfits/core.pyc in __init__(self, input, tbtype)
>     4887                     if col<= _nfields and col>  0:
>     4888                         cname = _commonNames[_keyNames.index(keyword)]
> ->  4889                         dict[col-1][cname] = _card.value
>     4890
>     4891             # data reading will be delayed
>
>
> /usr/local/lib/python2.6/dist-packages/pyfits/core.pyc in __getattr__(self, name)
>      438             self._extractKey()
>      439         elif name in ['value', 'comment']:
> -->  440             self._extractValueComment(name)
>      441         else:
>      442             raise AttributeError, name
>
> /usr/local/lib/python2.6/dist-packages/pyfits/core.pyc in _extractValueComment(self, name)
>     1450
>     1451                 # drop the ending "&"
>
> ->  1452                 if _val[-1] == '&':
>     1453                     _val = _val[:-1]
>     1454                 longstring = longstring + _val
>
> IndexError: string index out of range
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>    


-- 
Michael Droettboom
Science Software Branch
Space Telescope Science Institute
Baltimore, Maryland, USA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: pyfits_continue_workaround.diff
Type: text/x-patch
Size: 452 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/astropy/attachments/20101116/b0d5bfeb/attachment.bin>