[AstroPy] astropy.table.Table problems with VOTable input and FITS output

Doug Burke dburke.gw at gmail.com
Wed Apr 29 10:45:35 EDT 2020


Rick,

I've seen this when dealing with a VOTable which has variable-length string
columns. The column is stored as an object rather than a string, and the
FITS serializer can't handle this.

You can see the affected columns with something like:

In [14]: for col in tab.columns:
    ...:     if tab[col].dtype == object: print(col)
    ...:
name
planet_status
updated
publication
detection_type
mass_detection_type
radius_detection_type
alternate_names
molecules
star_name
star_sp_type
star_detected_disc
star_magnetic_field
star_alternate_names

I worked around this by manually converting to string values - e.g.
something like the following, but there may be a nicer way to do this:

In [20]: for col in tab.columns:
    ...:     if tab[col].dtype != object: continue
    ...:     tab[col] = [str(n) for n in tab[col]]
    ...:

However, in this case it just leads to more errors - e.g.

In [21]: tab.write('delme.fits')
WARNING: table contains column(s) with defined 'format', 'description', or
'meta' info attributes. These will be dropped unless you install PyYAML.
[astropy.io.fits.connect]
---------------------------------------------------------------------------
UnitScaleError                            Traceback (most recent call last)
~/anaconda/envs/sherpa-sample-flux/lib/python3.8/site-packages/astropy/io/fits/convenience.py
in table_to_hdu(table, character_as_bytes)
    517             try:
--> 518                 col.unit = unit.to_string(format='fits')
    519             except UnitScaleError:

~/anaconda/envs/sherpa-sample-flux/lib/python3.8/site-packages/astropy/units/core.py
in to_string(self, format)
    604         f = unit_format.get_format(format)
--> 605         return f.to_string(self)
    606

~/anaconda/envs/sherpa-sample-flux/lib/python3.8/site-packages/astropy/units/format/fits.py
in to_string(cls, unit)
    120             if base % 1.0 != 0.0:
--> 121                 raise core.UnitScaleError(
    122                     "The FITS unit format is not able to represent
scales "

UnitScaleError: The FITS unit format is not able to represent scales that
are not powers of 10.  Multiply your data by 1.898125e+27.

During handling of the above exception, another exception occurred:

UnitScaleError                            Traceback (most recent call last)
<ipython-input-21-721daad32140> in <module>
----> 1 tab.write('delme.fits')

... and some more

On Wed, Apr 29, 2020 at 10:34 AM Frederic V. Hessman <
hessman at astro.physik.uni-goettingen.de> wrote:

> I'm trying to read tables exported by exoplanet.eu (exoplanet databse
> portal) in VOTable format.
>
> Here's a mini 1-row table in their format:
>
>
> astropy.table.Table didn't like the extra "description" attribute for the
> <RESOURCE> tag used by exoplanet.eu
>
> <RESOURCE description="Exoplanet ...">
>
> (the description should have been placed as an extra daughter tag) but
> that was easily removed.  Thereafter, I got this error when trying to
> re-write it as a FITS table (where I can store metadata - the old problem
> with astropy VOTables...):
>
> Python 3.5.9 (default, Nov  2 2019, 03:07:41)
> [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
> >>> from astropy.table import Table
> >>> tab = Table.read ('test.vot',format='votable')
> >>> tab.write ('test.fits',format='fits',overwrite=True)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/astropy/table/connect.py",
> line 114, in __call__
>     registry.write(instance, *args, **kwargs)
>   File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/astropy/io/registry.py",
> line 566, in write
>     writer(data, *args, **kwargs)
>   File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/astropy/io/fits/connect.py",
> line 387, in write_table_fits
>     table_hdu = table_to_hdu(input, character_as_bytes=True)
>   File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/astropy/io/fits/convenience.py",
> line 480, in table_to_hdu
>     table_hdu = BinTableHDU.from_columns(np.array(table.filled()),
> header=hdr, character_as_bytes=True)
>   File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/astropy/io/fits/hdu/table.py",
> line 125, in from_columns
>     coldefs = cls._columns_type(columns)
>   File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/astropy/io/fits/column.py",
> line 1375, in __init__
>     self._init_from_array(input)
>   File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/astropy/io/fits/column.py",
> line 1410, in _init_from_array
>     format = self._col_format_cls.from_recformat(ftype)
>   File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/astropy/io/fits/column.py",
> line 273, in from_recformat
>     return cls(_convert_format(recformat, reverse=True))
>   File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/astropy/io/fits/column.py",
> line 2401, in _convert_format
>     return _convert_record2fits(format)
>   File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/astropy/io/fits/column.py",
> line 2364, in _convert_record2fits
>     raise ValueError('Illegal format `{}`.'.format(format))
> ValueError: Illegal format `object`.
>
>
> Note that the table was initially input just fine: all of the data is
> there and accessible.  The problem was just in storing it in FITS format.
>
> Any ideas?
>
> Rick
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20200429/49e1e5aa/attachment.html>


More information about the AstroPy mailing list