[AstroPy] Fixing broken FITS headers

[Demitri Muna]

Hi Rick,

On Aug 17, 2016, at 6:30 AM, Frederic V. Hessman <hessman at astro.physik.uni-goettingen.de> wrote:

> Is there some easy way of forcing acceptance (so that I can repair the header) or replacing the codec?  Don’t see anything obvious in the API….

Not speaking as someone who wrote the software, but it wouldn't be an easy thing to do. The FITS format is extremely specific on a byte-to-byte basis, and trying to understand how a file is broken wouldn't be straightforward. I actually recommend such a program should not be part of astropy.io.fits but something standalone, but that's neither here nor there.

Since the problem is in the first header, it's easier to solve. The format of the header is a series of 80 byte (80 character) cards. The last one has a keyword of "END" followed by 77 spaces. There may be additional empty (spaces) 80 character blocks after that before the data starts.

To fix the file, I'd open it as a binary file and begin reading it 80 bytes at a time. Cast those bytes to a UTF-8 string. One of those "lines" will contain your umlaut (hopefully there is just one?). If the line doesn't contain the umlaut, write it back (as binary!) out to a new file. Replace the umlaut character with something valid in the string and write it to the new file. Now just read the rest of the file from that point and write it directly to the new file.

If there is more than one umlaut you just need to repeat this. Remember that the character is two bytes, so the line that contains it is one character short from what the actual header line is. It will be a matter of bookkeeping then. This may sound complicated, but it should be pretty straightforward.


Do you know what software wrote that file? I'd strongly argue that any FITS writer should not output invalid files in the first place. My guess is that it came from IDL, which as far as FITS writers go, is the wild west.

Cheers,
Demitri

_________________________________________
Demitri Muna
http://muna.com

Department of Astronomy
La Ohio State University

My Projects:
http://nightlightapp.io
http://trillianverse.org
http://scicoder.org