[AstroPy] Accessing a FITS table generates "ValueError: Inconsistent data column lengths: set([0, 27645])"
Jordan Alexander
jordan.alexander at aut.ac.nz
Tue Apr 17 18:59:11 EDT 2018
Your "tested" work-around, Tom:
hdulist.info()
dat = hdulist[8].data
names = [name for name in dat.dtype.names if name != 'col_with_0J']
cols = [dat[name] for name in names]
tbl = Table(cols, names=names)
<generates>
Filename: WS001H.0.bin0000.source0000.FITS
No. Name Ver Type Cards Dimensions Format
0 PRIMARY 1 GroupsHDU 51 () 0 Groups 0 Parameters
1 ARRAY_GEOMETRY 1 BinTableHDU 53 2R x 7C ['8A', '3D', '3E', '0D', '1J', '1J', '3E']
2 SOURCE 1 BinTableHDU 89 3R x 26C [1J, 16A, 1J, 4A, 1J, 8E, 8E, 8E, 8E, 8E, 8D, 1D, 1D, 8A, 1D, 1D, 8D, 8A, 8A, 8D, 1D, 1D, 1E, 1D, 1D, 1D]
3 ANTENNA 1 BinTableHDU 52 18R x 13C [1D, 1E, 8A, 1J, 1J, 1J, 1J, 1A, 8E, 8E, 1A, 8E, 8E]
4 FREQUENCY 1 BinTableHDU 35 1R x 6C [1J, 8D, 8E, 8E, 8J, 8J]
5 INTERFEROMETER_MODEL 1 BinTableHDU 83 946R x 20C [1D, 1E, 1J, 1J, 1J, 1J, 1E, 8E, 48D, 6D, 48D, 6D, 1E, 1E, 48D, 6D, 48D, 6D, 1E, 1E]
6 CALC 1 BinTableHDU 75 5R x 11C [1D, 1D, 1D, 1D, 1A, 2D, 1A, 1D, 1D, 1D, 1D]
7 MODEL_COMPS 1 BinTableHDU 86 946R x 21C [1D, 1J, 1J, 1J, 1J, 1D, 1D, 1D, 1D, 1D, 1D, 8E, 8E, 1E, 1E, 1D, 1D, 8E, 8E, 1E, 1E]
8 UV_DATA 1 BinTableHDU 93 27645R x 13C [1E, 1E, 1E, 1D, 1D, 1J, 1J, 1J, 1J, 1E, 32E, 0J, 4096E]
9 PHASE-CAL 1 BinTableHDU 61 2406R x 17C [1D, 1E, 1J, 1J, 1J, 1J, 1D, 32E, 16D, 16E, 16E, 16E, 32E, 16D, 16E, 16E, 16E]
10 GAIN_CURVE 1 BinTableHDU 62 0R x 19C [1J, 1J, 1J, 8J, 8J, 8J, 8J, 8E, 48E, 48E, 8E, 8J, 8J, 8J, 8J, 8E, 48E, 48E, 8E]
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-100-08709251a86a> in <module>()
9 cols = [dat[name] for name in names]
10
---> 11 tbl = Table(cols, names=names)
/usr/local/lib/python2.7/dist-packages/astropy/table/table.pyc in __init__(self, data, masked, names, dtype, meta, copy, rows, copy_indices, **kwargs)
411 init_func(data, names, dtype, n_cols, copy)
412
--> 413 # Whatever happens above, the masked property should be set to a boolean
414 if type(self.masked) is not bool:
415 raise TypeError("masked property has not been set to True or False")
/usr/local/lib/python2.7/dist-packages/astropy/table/table.pyc in _init_from_list(self, data, names, dtype, n_cols, copy)
684 self._init_from_cols(cols)
685
--> 686 def _init_from_ndarray(self, data, names, dtype, n_cols, copy):
687 """Initialize table from an ndarray structured array"""
688
/usr/local/lib/python2.7/dist-packages/astropy/table/table.pyc in _init_from_cols(self, cols)
749 .format(lengths))
750
--> 751 # Set the table masking
752 self._set_masked_from_cols(cols)
753
ValueError: Inconsistent data column lengths: set([0, 27645])
________________________________
From: AstroPy [astropy-bounces+jordan.alexander=aut.ac.nz at python.org] on behalf of Aldcroft, Thomas [aldcroft at head.cfa.harvard.edu]
Sent: 18 April 2018 00:45
To: Astronomical Python mailing list
Subject: Re: [AstroPy] Accessing a FITS table generates "ValueError: Inconsistent data column lengths: set([0, 27645])"
Hi Jordan,
That particular HDU (#8) has a table with a column that has data type 0J. This the source of the problem. I see that numpy arrays can have a length with zero-width (e.g. shape=(10, 0)), so in theory astropy should be able to support this. If you can file this as an issue on GitHub that would be very helpful.
In the meantime something like this might work (untested) to just skip that column:
dat = hdulist[8].data
names = [name for name in dat.dtype.names if name != 'col_with_0J']
cols = [dat[name] for name in names]
tbl = Table(cols, names=names)
- Tom
On Tue, Apr 17, 2018 at 6:53 AM, Jordan Alexander <jordan.alexander at aut.ac.nz<mailto:jordan.alexander at aut.ac.nz>> wrote:
First, thank you astropy for this outstanding package. It played an important role in the completion of my PhD; much respect :)
Using python 2.7.12 & astropy 2.0.5 and following Lia R. Corrales' useful tutorial,
http://www.astropy.org/astropy-tutorials/FITS-tables.html
I am trying to access data stored in a FITS table, which generates
ValueError: Inconsistent data column lengths: set([0, 27645])"
Below are my steps and intermediate results that leads to this error...
Can you please help here, astropy at python.org<mailto:astropy at python.org>?
P.S. I am able to access other tables in the following FITS file, just not the one I am interested in...
--------
from astropy.io<http://astropy.io> import fits
from astropy.table import Table
hdulist = fits.open('WS001H.0.bin0000.source0000.FITS')
hdulist.info<http://hdulist.info>()
<generates the following output...>
Filename: WS001H.0.bin0000.source0000.FITS
No. Name Ver Type Cards Dimensions Format
0 PRIMARY 1 GroupsHDU 51 () 0 Groups 0 Parameters
1 ARRAY_GEOMETRY 1 BinTableHDU 53 2R x 7C [8A, 3D, 3E, 0D, 1J, 1J, 3E]
2 SOURCE 1 BinTableHDU 89 3R x 26C [1J, 16A, 1J, 4A, 1J, 8E, 8E, 8E, 8E, 8E, 8D, 1D, 1D, 8A, 1D, 1D, 8D, 8A, 8A, 8D, 1D, 1D, 1E, 1D, 1D, 1D]
3 ANTENNA 1 BinTableHDU 52 18R x 13C [1D, 1E, 8A, 1J, 1J, 1J, 1J, 1A, 8E, 8E, 1A, 8E, 8E]
4 FREQUENCY 1 BinTableHDU 35 1R x 6C [1J, 8D, 8E, 8E, 8J, 8J]
5 INTERFEROMETER_MODEL 1 BinTableHDU 83 946R x 20C [1D, 1E, 1J, 1J, 1J, 1J, 1E, 8E, 48D, 6D, 48D, 6D, 1E, 1E, 48D, 6D, 48D, 6D, 1E, 1E]
6 CALC 1 BinTableHDU 75 5R x 11C ['1D', '1D', '1D', '1D', '1A', '2D', '1A', '1D', '1D', '1D', '1D']
7 MODEL_COMPS 1 BinTableHDU 86 946R x 21C ['1D', '1J', '1J', '1J', '1J', '1D', '1D', '1D', '1D', '1D', '1D', '8E', '8E', '1E', '1E', '1D', '1D', '8E', '8E', '1E', '1E']
8 UV_DATA 1 BinTableHDU 93 27645R x 13C ['1E', '1E', '1E', '1D', '1D', '1J', '1J', '1J', '1J', '1E', '32E', '0J', '4096E']
9 PHASE-CAL 1 BinTableHDU 61 2406R x 17C [1D, 1E, 1J, 1J, 1J, 1J, 1D, 32E, 16D, 16E, 16E, 16E, 32E, 16D, 16E, 16E, 16E]
10 GAIN_CURVE 1 BinTableHDU 62 0R x 19C [1J, 1J, 1J, 8J, 8J, 8J, 8J, 8E, 48E, 48E, 8E, 8J, 8J, 8J, 8J, 8E, 48E, 48E, 8E]
Table(hdulist[8].data)
<generates...>
ValueError Traceback (most recent call last)
<ipython-input-82-fca4b3ff146b> in <module>()
1 hdulist.info()
----> 2 Table(hdulist[8].data)
/usr/local/lib/python2.7/dist-packages/astropy/table/table.pyc in __init__(self, data, masked, names, dtype, meta, copy, rows, copy_indices, **kwargs)
411 init_func(data, names, dtype, n_cols, copy)
412
--> 413 # Whatever happens above, the masked property should be set to a boolean
414 if type(self.masked) is not bool:
415 raise TypeError("masked property has not been set to True or False")
/usr/local/lib/python2.7/dist-packages/astropy/table/table.pyc in _init_from_ndarray(self, data, names, dtype, n_cols, copy)
700 self._init_from_list(cols, names, dtype, n_cols, copy)
701 else:
--> 702 dtype = [(name, col.dtype, col.shape[1:]) for name, col in zip(names, cols)]
703 newdata = data.view(dtype).ravel()
704 columns = self.TableColumns()
/usr/local/lib/python2.7/dist-packages/astropy/table/table.pyc in _init_from_list(self, data, names, dtype, n_cols, copy)
684 self._init_from_cols(cols)
685
--> 686 def _init_from_ndarray(self, data, names, dtype, n_cols, copy):
687 """Initialize table from an ndarray structured array"""
688
/usr/local/lib/python2.7/dist-packages/astropy/table/table.pyc in _init_from_cols(self, cols)
749 .format(lengths))
750
--> 751 # Set the table masking
752 self._set_masked_from_cols(cols)
753
ValueError: Inconsistent data column lengths: set([0, 27645])
_______________________________________________
AstroPy mailing list
AstroPy at python.org<mailto:AstroPy at python.org>
https://mail.python.org/mailman/listinfo/astropy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20180417/aeeb75ee/attachment-0001.html>
More information about the AstroPy
mailing list