[AstroPy] dropped blanks in FITS string header cards

[Hessman, Frederic]

I'm parsing IRAF multispec headers by hand (specutils has problems with my files ☹️) and came across this apparent bug:

multispec stores lots and lots of dispersion info in a long string instead of putting it into a reasonable set of cards (trying to use less 8-track tape?).  The string is so long that it has to be distributed over several cards.  That means that a string containing a list of numbers is split into sub-strings and placed in adjacent cards.  The spaces in the strings are important - they separate the string representations of numbers - but they appear at random positions in the sub-strings.

While parsing a FITS file with a header containing multispec info, I couldn't extract all of the information because the concatenated dispersion info was missing an internal space used to separate two numbers.

Here's an example of the effect:  first I set the values of the cards as strings:

>>> WAT2_001 = 'wtype=multispec spec1 = "1 1 2 30. 3.4938694016091 1989 -8.060336767'
>>> WAT2_002 = '2905E-5 315.89 325.25 1. 0. 2 6 14.65 1972.02 6375.15322418564 589.9'
>>> WAT2_003 = '46425975261 -0.582739656692844 1.02948926318171 -0.0423755076218827 '
>>> WAT2_004 = '0.00126765958623805"'

When I print them out, everything is OK, of course:

>>> for key in ['WAT2_001','WAT2_002','WAT2_003','WAT2_004'] :
...     val = globals()[key]
...     print (f'{key} = \'{val}\'')
WAT2_001 = 'wtype=multispec spec1 = "1 1 2 30. 3.4938694016091 1989 -8.060336767'
WAT2_002 = '2905E-5 315.89 325.25 1. 0. 2 6 14.65 1972.02 6375.15322418564 589.9'
WAT2_003 = '46425975261 -0.582739656692844 1.02948926318171 -0.0423755076218827 '
WAT2_004 = '0.00126765958623805"'

So far, so good.  Note that multispec uses " to define an internal string within the concatenated information, which means that there's a single " in the middle of the 1st card and a 2nd one at the end of the 4th card.  Note also that there are several numbers that have been split between cards.

Now, put this info into a HDU and see what happens:

>>> hdu = fits.PrimaryHDU(np.arange(100.0))
>>> hdr = hdu.header
>>> for key in ['WAT2_001','WAT2_002','WAT2_003','WAT2_004'] :
...    hdr[key] = globals()[key]
...    print (f'{key} = \'{hdr[key]}\'')
WAT2_001 = 'wtype=multispec spec1 = "1 1 2 30. 3.4938694016091 1989 -8.060336767'
WAT2_002 = '2905E-5 315.89 325.25 1. 0. 2 6 14.65 1972.02 6375.15322418564 589.9'
WAT2_003 = '46425975261 -0.582739656692844 1.02948926318171 -0.0423755076218827'
WAT2_004 = '0.00126765958623805"'

i.e. the space at the end of the 3rd card is dropped, later creating the unparseable number "-0.04237550762188270.00126765958623805" in the concatenated string.  This is a random error in practice, since it depends upon having a space appear in the original string at the problematic position.

If one sticks in other content, it always happens whenever the string has a trailing blank, no matter what position or length.

Looks like a bug.  Using Python 3.9.11 under OSX 12.6.1 and astropy 5.3.3

Rick

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/astropy/attachments/20231010/69069db4/attachment.html>