[AstroPy] Accessing a FITS table generates "ValueError: Inconsistent data column lengths: set([0, 27645])"

[Derek Homeier]

On 17. Apr 2018, at 12:53 PM, Jordan Alexander <jordan.alexander at aut.ac.nz> wrote:
> 
> First, thank you astropy for this outstanding package.  It played an important role in the completion of my PhD; much respect :)
> 
> Using python 2.7.12 &  astropy 2.0.5 and following Lia R. Corrales' useful tutorial,
> 
> http://www.astropy.org/astropy-tutorials/FITS-tables.html
> 
> I am trying to access data stored in a FITS table, which generates 
> 
> ValueError: Inconsistent data column lengths: set([0, 27645])"
> 
> Below are my steps and intermediate results that leads to this error...
> 
> Can you please help here, astropy at python.org?
> 
> P.S. I am able to access other tables in the following FITS file, just not the one I am interested in...
> 
> --------
> 
> from astropy.io import fits
> 
> from astropy.table import Table
> 
> hdulist = fits.open('WS001H.0.bin0000.source0000.FITS')
> 
> hdulist.info()
> 
> <generates the following output...>
> Filename: WS001H.0.bin0000.source0000.FITS
> No.    Name      Ver    Type      Cards   Dimensions   Format
>   0  PRIMARY       1 GroupsHDU       51   ()      0 Groups  0 Parameters
>   1  ARRAY_GEOMETRY    1 BinTableHDU     53   2R x 7C   [8A, 3D, 3E, 0D, 1J, 1J, 3E]   
>   2  SOURCE        1 BinTableHDU     89   3R x 26C   [1J, 16A, 1J, 4A, 1J, 8E, 8E, 8E, 8E, 8E, 8D, 1D, 1D, 8A, 1D, 1D, 8D, 8A, 8A, 8D, 1D, 1D, 1E, 1D, 1D, 1D]   
>   3  ANTENNA       1 BinTableHDU     52   18R x 13C   [1D, 1E, 8A, 1J, 1J, 1J, 1J, 1A, 8E, 8E, 1A, 8E, 8E]   
>   4  FREQUENCY     1 BinTableHDU     35   1R x 6C   [1J, 8D, 8E, 8E, 8J, 8J]   
>   5  INTERFEROMETER_MODEL    1 BinTableHDU     83   946R x 20C   [1D, 1E, 1J, 1J, 1J, 1J, 1E, 8E, 48D, 6D, 48D, 6D, 1E, 1E, 48D, 6D, 48D, 6D, 1E, 1E]   
>   6  CALC          1 BinTableHDU     75   5R x 11C   ['1D', '1D', '1D', '1D', '1A', '2D', '1A', '1D', '1D', '1D', '1D']   
>   7  MODEL_COMPS    1 BinTableHDU     86   946R x 21C   ['1D', '1J', '1J', '1J', '1J', '1D', '1D', '1D', '1D', '1D', '1D', '8E', '8E', '1E', '1E', '1D', '1D', '8E', '8E', '1E', '1E']   
>   8  UV_DATA       1 BinTableHDU     93   27645R x 13C   ['1E', '1E', '1E', '1D', '1D', '1J', '1J', '1J', '1J', '1E', '32E', '0J', '4096E']   
> 
That’s a somewhat unusual, if not pathological, table shape in HDU 8 - column 12 has zero width, i.e. shape (0, nrow).
If that’s actually allowed by the FITS standard, it might be a bug, as astropy apparently converts this into a column of
length 0 (instead of a shape=(0, ), length nrow=27645 column) which then fails in initialising the table.
Can you read hdulist[1].data (which has a zero width column as well)?

Cheers,
				Derek