[AstroPy] Accessing a FITS table generates "ValueError: Inconsistent data column lengths: set([0, 27645])"

[Aldcroft, Thomas]

Hi Jordan,

That particular HDU (#8) has a table with a column that has data type 0J.
This the source of the problem.  I see that numpy arrays can have a length
with zero-width (e.g. shape=(10, 0)), so in theory astropy should be able
to support this.  If you can file this as an issue on GitHub that would be
very helpful.

In the meantime something like this might work (untested) to just skip that
column:

dat = hdulist[8].data
names = [name for name in dat.dtype.names if name != 'col_with_0J']
cols = [dat[name] for name in names]

tbl = Table(cols, names=names)

- Tom

On Tue, Apr 17, 2018 at 6:53 AM, Jordan Alexander <
jordan.alexander at aut.ac.nz> wrote:

> First, thank you astropy for this outstanding package.  It played an
> important role in the completion of my PhD; much respect :)
>
> Using python 2.7.12 &  astropy 2.0.5 and following Lia R. Corrales' useful
> tutorial,
>
> http://www.astropy.org/astropy-tutorials/FITS-tables.html
>
> I am trying to access data stored in a FITS table, which generates
>
> ValueError: Inconsistent data column lengths: set([0, 27645])"
>
> Below are my steps and intermediate results that leads to this error...
>
> Can you please help here, astropy at python.org?
>
> P.S. I am able to access other tables in the following FITS file, just not
> the one I am interested in...
>
> --------
>
> from astropy.io import fits
>
> from astropy.table import Table
>
> hdulist = fits.open('WS001H.0.bin0000.source0000.FITS')
>
> hdulist.info()
>
> <generates the following output...>
>
> Filename: WS001H.0.bin0000.source0000.FITS
> No.    Name      Ver    Type      Cards   Dimensions   Format
>   0  PRIMARY       1 GroupsHDU       51   ()      0 Groups  0 Parameters
>   1  ARRAY_GEOMETRY    1 BinTableHDU     53   2R x 7C   [8A, 3D, 3E, 0D, 1J, 1J, 3E]
>   2  SOURCE        1 BinTableHDU     89   3R x 26C   [1J, 16A, 1J, 4A, 1J, 8E, 8E, 8E, 8E, 8E, 8D, 1D, 1D, 8A, 1D, 1D, 8D, 8A, 8A, 8D, 1D, 1D, 1E, 1D, 1D, 1D]
>   3  ANTENNA       1 BinTableHDU     52   18R x 13C   [1D, 1E, 8A, 1J, 1J, 1J, 1J, 1A, 8E, 8E, 1A, 8E, 8E]
>   4  FREQUENCY     1 BinTableHDU     35   1R x 6C   [1J, 8D, 8E, 8E, 8J, 8J]
>   5  INTERFEROMETER_MODEL    1 BinTableHDU     83   946R x 20C   [1D, 1E, 1J, 1J, 1J, 1J, 1E, 8E, 48D, 6D, 48D, 6D, 1E, 1E, 48D, 6D, 48D, 6D, 1E, 1E]
>   6  CALC          1 BinTableHDU     75   5R x 11C   ['1D', '1D', '1D', '1D', '1A', '2D', '1A', '1D', '1D', '1D', '1D']
>   7  MODEL_COMPS    1 BinTableHDU     86   946R x 21C   ['1D', '1J', '1J', '1J', '1J', '1D', '1D', '1D', '1D', '1D', '1D', '8E', '8E', '1E', '1E', '1D', '1D', '8E', '8E', '1E', '1E']
>   8  UV_DATA       1 BinTableHDU     93   27645R x 13C   ['1E', '1E', '1E', '1D', '1D', '1J', '1J', '1J', '1J', '1E', '32E', '0J', '4096E']
>   9  PHASE-CAL     1 BinTableHDU     61   2406R x 17C   [1D, 1E, 1J, 1J, 1J, 1J, 1D, 32E, 16D, 16E, 16E, 16E, 32E, 16D, 16E, 16E, 16E]
>  10  GAIN_CURVE    1 BinTableHDU     62   0R x 19C   [1J, 1J, 1J, 8J, 8J, 8J, 8J, 8E, 48E, 48E, 8E, 8J, 8J, 8J, 8J, 8E, 48E, 48E, 8E]
>
>
> Table(hdulist[8].data)
>
> <generates...>
>
> ValueError                                Traceback (most recent call last)<ipython-input-82-fca4b3ff146b> in <module>()      1 hdulist.info()----> 2 Table(hdulist[8].data)
> /usr/local/lib/python2.7/dist-packages/astropy/table/table.pyc in __init__(self, data, masked, names, dtype, meta, copy, rows, copy_indices, **kwargs)    411         init_func(data, names, dtype, n_cols, copy)    412 --> 413         # Whatever happens above, the masked property should be set to a boolean    414         if type(self.masked) is not bool:    415             raise TypeError("masked property has not been set to True or False")
> /usr/local/lib/python2.7/dist-packages/astropy/table/table.pyc in _init_from_ndarray(self, data, names, dtype, n_cols, copy)    700             self._init_from_list(cols, names, dtype, n_cols, copy)    701         else:--> 702             dtype = [(name, col.dtype, col.shape[1:]) for name, col in zip(names, cols)]    703             newdata = data.view(dtype).ravel()    704             columns = self.TableColumns()
> /usr/local/lib/python2.7/dist-packages/astropy/table/table.pyc in _init_from_list(self, data, names, dtype, n_cols, copy)    684         self._init_from_cols(cols)    685 --> 686     def _init_from_ndarray(self, data, names, dtype, n_cols, copy):    687         """Initialize table from an ndarray structured array"""    688
> /usr/local/lib/python2.7/dist-packages/astropy/table/table.pyc in _init_from_cols(self, cols)    749                              .format(lengths))    750 --> 751         # Set the table masking    752         self._set_masked_from_cols(cols)    753
> ValueError: Inconsistent data column lengths: set([0, 27645])
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20180417/c099be42/attachment-0001.html>