[AstroPy] Accessing a FITS table generates "ValueError: Inconsistent data column lengths: set([0, 27645])"
Jordan Alexander
jordan.alexander at aut.ac.nz
Tue Apr 17 18:52:19 EDT 2018
>Could you make one more test and check what happens if you try
>tbl = Table(np.array(hdulist[1].data)) ?
Here's what happens, Derek:
hdulist.info()
Table(np.array(hdulist[1].data))
<generates>
Filename: WS001H.0.bin0000.source0000.FITS
No. Name Ver Type Cards Dimensions Format
0 PRIMARY 1 GroupsHDU 51 () 0 Groups 0 Parameters
1 ARRAY_GEOMETRY 1 BinTableHDU 53 2R x 7C ['8A', '3D', '3E', '0D', '1J', '1J', '3E']
2 SOURCE 1 BinTableHDU 89 3R x 26C [1J, 16A, 1J, 4A, 1J, 8E, 8E, 8E, 8E, 8E, 8D, 1D, 1D, 8A, 1D, 1D, 8D, 8A, 8A, 8D, 1D, 1D, 1E, 1D, 1D, 1D]
3 ANTENNA 1 BinTableHDU 52 18R x 13C [1D, 1E, 8A, 1J, 1J, 1J, 1J, 1A, 8E, 8E, 1A, 8E, 8E]
4 FREQUENCY 1 BinTableHDU 35 1R x 6C [1J, 8D, 8E, 8E, 8J, 8J]
5 INTERFEROMETER_MODEL 1 BinTableHDU 83 946R x 20C [1D, 1E, 1J, 1J, 1J, 1J, 1E, 8E, 48D, 6D, 48D, 6D, 1E, 1E, 48D, 6D, 48D, 6D, 1E, 1E]
6 CALC 1 BinTableHDU 75 5R x 11C [1D, 1D, 1D, 1D, 1A, 2D, 1A, 1D, 1D, 1D, 1D]
7 MODEL_COMPS 1 BinTableHDU 86 946R x 21C [1D, 1J, 1J, 1J, 1J, 1D, 1D, 1D, 1D, 1D, 1D, 8E, 8E, 1E, 1E, 1D, 1D, 8E, 8E, 1E, 1E]
8 UV_DATA 1 BinTableHDU 93 27645R x 13C [1E, 1E, 1E, 1D, 1D, 1J, 1J, 1J, 1J, 1E, 32E, 0J, 4096E]
9 PHASE-CAL 1 BinTableHDU 61 2406R x 17C [1D, 1E, 1J, 1J, 1J, 1J, 1D, 32E, 16D, 16E, 16E, 16E, 32E, 16D, 16E, 16E, 16E]
10 GAIN_CURVE 1 BinTableHDU 62 0R x 19C [1J, 1J, 1J, 8J, 8J, 8J, 8J, 8E, 48E, 48E, 8E, 8J, 8J, 8J, 8J, 8E, 48E, 48E, 8E]
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
/usr/local/lib/python2.7/dist-packages/IPython/core/formatters.pyc in __call__(self, obj)
697 type_pprinters=self.type_printers,
698 deferred_pprinters=self.deferred_printers)
--> 699 printer.pretty(obj)
700 printer.flush()
701 return stream.getvalue()
/usr/local/lib/python2.7/dist-packages/IPython/lib/pretty.pyc in pretty(self, obj)
401 if cls is not object \
402 and callable(cls.__dict__.get('__repr__')):
--> 403 return _repr_pprint(obj, self, cycle)
404
405 return _default_pprint(obj, self, cycle)
/usr/local/lib/python2.7/dist-packages/IPython/lib/pretty.pyc in _repr_pprint(obj, p, cycle)
701 """A pprint that just redirects to the normal repr function."""
702 # Find newlines and replace them with p.break_()
--> 703 output = repr(obj)
704 for idx,output_line in enumerate(output.splitlines()):
705 if idx:
/usr/local/lib/python2.7/dist-packages/astropy/table/table.pyc in __repr__(self)
873 return self._base_repr_(html=False, max_width=None)
874
--> 875 def __unicode__(self):
876 return '\n'.join(self.pformat())
877 if not six.PY2:
/usr/local/lib/python2.7/dist-packages/astropy/table/table.pyc in _base_repr_(self, html, descr_vals, max_width, tableid, show_dtype, max_lines, tableclass)
860 max_lines=max_lines, tableclass=tableclass)
861
--> 862 out = descr + '\n'.join(data_lines)
863 if six.PY2 and isinstance(out, six.text_type):
864 out = out.encode('utf-8')
/usr/local/lib/python2.7/dist-packages/astropy/table/pprint.pyc in _pformat_table(self, table, max_lines, max_width, show_name, show_unit, show_dtype, html, tableid, tableclass, align)
539
540 if not cols:
--> 541 return ['<No columns>'], {'show_length': False}
542
543 # Use the values for the last column since they are all the same
/usr/local/lib/python2.7/dist-packages/astropy/table/pprint.pyc in _pformat_col(self, col, max_lines, show_name, show_unit, show_dtype, show_length, html, align)
252
253 if html:
--> 254 from ..utils.xml.writer import xml_escape
255 n_header = outs['n_header']
256 for i, col_str in enumerate(col_strs):
/usr/local/lib/python2.7/dist-packages/astropy/table/pprint.pyc in _pformat_col_iter(self, col, max_lines, show_name, show_unit, outs, show_dtype, show_length)
450 yield col_str
451
--> 452 outs['show_length'] = show_length
453 outs['n_header'] = n_header
454 outs['i_centers'] = i_centers
astropy/table/_column_mixins.pyx in astropy.table._column_mixins._ColumnGetitemShim.__getitem__()
astropy/table/_column_mixins.pyx in astropy.table._column_mixins.base_getitem()
astropy/table/_column_mixins.pyx in astropy.table._column_mixins.column_getitem()
IndexError: index 0 is out of bounds for axis 1 with size 0
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
/usr/local/lib/python2.7/dist-packages/IPython/core/formatters.pyc in __call__(self, obj)
336 method = get_real_method(obj, self.print_method)
337 if method is not None:
--> 338 return method()
339 return None
340 else:
/usr/local/lib/python2.7/dist-packages/astropy/table/table.pyc in _repr_html_(self)
870 tableclass=conf.default_notebook_table_class)
871
--> 872 def __repr__(self):
873 return self._base_repr_(html=False, max_width=None)
874
/usr/local/lib/python2.7/dist-packages/astropy/table/table.pyc in _base_repr_(self, html, descr_vals, max_width, tableid, show_dtype, max_lines, tableclass)
860 max_lines=max_lines, tableclass=tableclass)
861
--> 862 out = descr + '\n'.join(data_lines)
863 if six.PY2 and isinstance(out, six.text_type):
864 out = out.encode('utf-8')
/usr/local/lib/python2.7/dist-packages/astropy/table/pprint.pyc in _pformat_table(self, table, max_lines, max_width, show_name, show_unit, show_dtype, html, tableid, tableclass, align)
539
540 if not cols:
--> 541 return ['<No columns>'], {'show_length': False}
542
543 # Use the values for the last column since they are all the same
/usr/local/lib/python2.7/dist-packages/astropy/table/pprint.pyc in _pformat_col(self, col, max_lines, show_name, show_unit, show_dtype, show_length, html, align)
252
253 if html:
--> 254 from ..utils.xml.writer import xml_escape
255 n_header = outs['n_header']
256 for i, col_str in enumerate(col_strs):
/usr/local/lib/python2.7/dist-packages/astropy/table/pprint.pyc in _pformat_col_iter(self, col, max_lines, show_name, show_unit, outs, show_dtype, show_length)
450 yield col_str
451
--> 452 outs['show_length'] = show_length
453 outs['n_header'] = n_header
454 outs['i_centers'] = i_centers
astropy/table/_column_mixins.pyx in astropy.table._column_mixins._ColumnGetitemShim.__getitem__()
astropy/table/_column_mixins.pyx in astropy.table._column_mixins.base_getitem()
astropy/table/_column_mixins.pyx in astropy.table._column_mixins.column_getitem()
IndexError: index 0 is out of bounds for axis 1 with size 0
________________________________________
From: AstroPy [astropy-bounces+jordan.alexander=aut.ac.nz at python.org] on behalf of Derek Homeier [derek at astro.physik.uni-goettingen.de]
Sent: 18 April 2018 08:41
To: Astronomical Python mailing list
Subject: Re: [AstroPy] Accessing a FITS table generates "ValueError: Inconsistent data column lengths: set([0, 27645])"
On 17 Apr 2018, at 9:58 pm, Jordan Alexander <jordan.alexander at aut.ac.nz> wrote:
>
>> Can you read hdulist[1].data (which has a zero width column as well)?
>
> fails in the same way, as you predicted, given the zero width column.
>
> As suggested, I will file this issue with https://github.com/astropy/astropy/issues.
>
> The FITS file in question with tables containing zero-width columns that generates this issue is ~0.5Gbytes, so will try to find a smaller-sized example file....
>
> Will follow-up with your untested work-around, Tom...
Thanks for the follow-up, Jordan!
Could you make one more test and check what happens if you try
tbl = Table(np.array(hdulist[1].data)) ?
- hdulist[1].data is a FITS_rec, which is basically a container for a numpy record array,
but I am wondering if casting it to a regular (structured) ndarray will let it proceed as far
as creating the table, as it did in my tests.
For the example file, you might try
hdulist[:2].writeto(‘ValueError.fits’)
which should create a file from just the 0th and 1st HDU, sufficient to demonstrate the issue
(unless it already fails along the way on the 0-dim column…).
Best,
Derek
_______________________________________________
AstroPy mailing list
AstroPy at python.org
https://mail.python.org/mailman/listinfo/astropy
More information about the AstroPy
mailing list