[AstroPy] Convert FITS header into astropy Table entries?

[Erik Bray]

On 08/19/2015 10:36 AM, Jens Melinder wrote:
> Hi,
> there are a couple of problems with doing it that way I think.
>
> 1. Table will not like column names with the same name. For fits headers these
> are the HISTORY and COMMENT fields which will have multiple entries in the
> output list you get from h0.keys().

Fortunately though when you do header['COMMENT'] or header['HISTORY'] it 
automatically combines the values of all COMMENT or HISTORY cards into a single, 
newline-separated value, so that makes things a little simpler)

What's more of a problem if you have duplicates of any non-commentary keyword 
(ex: a duplicate OBSERVER keyword).  This *should* never be the case, but 
technically FITS allows it, though it's strongly discouraged.  In my example 
below I ignore that case since it's fairly rare.

> 2. While not 100% certain I think also the empty keys (which are used to split
> the HST fist headers into different sections) will cause problems.

Right--they're treated the same as COMMENT and HISTORY cards (their keyword is 
just the empty string like in your example below).

> 3. Finally, there is also a small syntax error in your call to Table. Instead of
> writing rows=h0.values() you should write rows = [h0.values()] (making it a list
> of lists)
>
> If you don’t need the HISTORY and COMMENT fields a simple fix maybe something
> like this:
> keys_cl=[]
> vals_cl=[]
> for ii in range(len(h0.keys())):
> if (h0.keys()[ii] != ’’)&(h0.keys()[ii] != ’HISTORY’)&(h0.keys()[ii] != ’COMMENT’):
> keys_cl.append(h0.keys()[ii])
> vals_cl.append(h0.values()[ii])
>
> tab = Table(rows=[vals_cl],names=keys_cl)

This example has a couple issues--for one header.keys() returns a generator so 
you can't just index it directly.  I think a simpler approach is:

 >>> unique_keys = sorted(set(header))
 >>> t = Table(rows=[[header[k] for k in unique_keys]], names=unique_keys)

If you wanted to exclude commentary keywords like '', 'COMMENT', and 'HISTORY' 
you might instead do:

 >>> unique_keys = sorted(key for key in header
...                      if key not in ('', 'COMMENT', 'HISTORY'))

This all also assumed that all the headers you're processing have the exact same 
set of keywords in them.  If that weren't the case you might have do take a 
subtly different approach to determining what columns your table should have 
(for example by looping over all headers first and adding their header keywords 
to a set of all known keywords).

Erik

> 19 aug 2015 kl. 16:00 skrev Jason Tumlinson <tumlinson at stsci.edu
> <mailto:tumlinson at stsci.edu>>:
>
>> Hi,
>> I am trying to harvest the information from a large number of FITS headers,
>> parse them and do statistics on values, in an efficient manner. I read in each
>> header in the usual fashion:
>>
>> h = fits.open('lbiy83i0q_x1d.fits’)
>> h0 = h[0].header
>>
>> My goal then is to get all the header info into an astropy Table such that the
>> table columns are given by the header keys and the values in the rows are
>> given by the values in each of 100+ headers. I tried this:
>>
>> In [38]: t = Table(rows=h0.values(), names=h0.keys())
>> ---------------------------------------------------------------------------
>> TypeError                                 Traceback (most recent call last)
>> <ipython-input-38-8a0ece2925d5>in <module>()
>> ----> 1t = Table(rows=h0.values(), names=h0.keys())
>>
>> /Users/tumlinson/Ureka/python/lib/python2.7/site-packages/astropy/table/table.pycin
>> __init__(self, data, masked, names, dtype, meta, copy, rows)
>>     240                 data = rows
>>     241 else:
>> --> 242rec_data = recarray_fromrecords(rows)
>>     243                 data = [rec_data[name] for name in rec_data.dtype.names]
>>     244
>>
>> /Users/tumlinson/Ureka/python/lib/python2.7/site-packages/astropy/table/np_utils.pycin
>> recarray_fromrecords(rec_list)
>>     604# to column arrays in the recarray module; it could be there is a better
>>     605# way
>> --> 606nfields = len(rec_list[0])
>>     607     obj = np.array(rec_list, dtype=object)
>>     608     array_list = [np.array(obj[..., i].tolist()) for i in range(nfields)]
>>
>> TypeError: object of type 'bool' has no len()
>>
>> to no avail. Is this possible, easy, or wrongheaded? Advise appreciated.
>>
>> Thanks,
>> JT
>>
>>
>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org <mailto:AstroPy at scipy.org>
>> http://mail.scipy.org/mailman/listinfo/astropy
>
> -----------------------------------
> Jens Melinder
> Department of Astronomy
> Stockholm University
> jens at astro.su.se <mailto:jens at astro.su.se>
> +46 706471856
>
>
>
>
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>