[AstroPy] "ASCII" tables that contain non-ASCII characters

[Derek Homeier]

On 25 Oct 2016, at 12:01 am, Nathan Goldbaum <nathan12343 at gmail.com> wrote:
> 
> I believe this is issue 2923:
> 
> https://github.com/astropy/astropy/issues/2923
> 
> On Mon, Oct 24, 2016 at 4:45 PM, Benjamin Alan Weaver <baweaver at lbl.gov> wrote:
> Hello y'all,
> 
> We are trying to read "ASCII" tables containing atomic line data
> provided by NIST.  When you request the line wavelength data in
> angstroms, NIST very helpfully labels the columns with the angstrom
> symbol (Å), which is not strictly part of the ASCII character set.
> 
> We can read these tables with Table.read() *and* the environment
> variable LANG=en_US.utf-8 set.  However, if LANG is not set,
> Table.read() fails to decode these files.
> 
> As far as I can tell the underlying read() function in astropy.io.ascii
> does not accept keywords related to the file encoding.
> 
> So two questions:
> 
> 1. Is the lack of an encoding keyword a bug that should be reported?
> 
> 2. Is there a workaround that does not rely on LANG being set?

A workaround that would at least get you away without manipulating the
environment outside Python would be

import locale
locale.setlocale(locale.LC_ALL, str(‘en_US.utf8’))

Cheers,
					Derek