[AstroPy] Write machine readable tables in Python?

Paul Kuin npkuin at gmail.com
Fri Apr 11 17:39:31 EDT 2014


Hi Leo,

I see that the APJ instructions only seem to refer the the machine readable
ascii format.  Originally the intention was to support both fits format and
ASCII. Fits data files would be listed in the 'file summary' but not need a
format section. CDS always has preferred ASCII only. With the demise of the
NASA Astronomical Data Center some ten years ago it seems that CDS has
pushed this through as a requirement to ApJ.

Anyway, I''ll see what can be done to improve things.  It may be easier to
get the formatting solved in astropy then add  a few extra C scripts from
CDS.

- Paul


On Fri, Apr 11, 2014 at 9:25 PM, Aldcroft, Thomas <
aldcroft at head.cfa.harvard.edu> wrote:

>
>
> On Fri, Apr 11, 2014 at 3:50 PM, Leo Singer <lsinger at caltech.edu> wrote:
>
>> Hi Paul,
>>
>> Well, that's another question: what format is easiest to use for most
>> readers to ingest published data into their own work? MRT can be read as
>> either fixed-width or whitespace-delimited ASCII, so it seems fine as a
>> least common denominator.
>>
>
> If you don't have need for the full metadata capabilities of MRT (aka CDS)
> then simple CSV or other character-delimited formats are much easier for
> reading.   Despite the name, machine readable tables are not especially
> easy to read for machines since the header format has a lot of flexibility
> and intricacy.
>
> For datasets with required keyword metadata and other unstructured
> comments you might consider IPAC for an ASCII table based interchange
> format.  It's fairly simple but does the job, and `io.ascii` can write this
> format.
>
> And of course FITS is pretty much universally accessible even though you
> lose the human-readability aspect.
>
> - Tom
>
>
>
>>
>> This particular dataset is a catalog of ~1000 simulated binary neutron
>> star mergers from a Monte Carlo study of sky localization accuracy with
>> Advanced LIGO and Virgo. Each row contains the parameters of the simulated
>> binary including component masses and spins, sky location, inclination,
>> luminosity distance, as well as parameters about the recovered signal, such
>> as the SNR in each detector, and area of the 90% credible region.
>>
>> Leo
>>
>> On Apr 11, 2014, at 12:20 PM, Paul Kuin <npkuin at gmail.com> wrote:
>>
>> Are you sure that you can't just send it as (a) FITS file(s) + a readme -
>> then you should not need the format tables at all?
>>
>>
>>  On Fri, Apr 11, 2014 at 8:09 PM, Paul Kuin <npkuin at gmail.com> wrote:
>>
>>> .I worked in with francois Ochsenbein of the CDS to pin down the
>>> original format. He developed some tools in C that are useful. One will
>>> build a format table for the data file. So the following scenario would
>>> seem to make sense. I will need to look all this up as it has been nearly
>>> 20 years ago. But basically we we built then the tools to do automatic
>>> reformatting, verification and ingest of the ascii tables in the archive.
>>> 1. write ascii tables using astropy; keep a list of header names.
>>> 2. reformat the tables to a standard form (I think there's a C script to
>>> do that.
>>> 3. run the format table generator; perhaps the barebones ReadMe file is
>>> better. (C script)
>>> Then you have the ascii tables, and Readme which will need some editing
>>> by hand to fill in the abstract and add notes and such.
>>>
>>> I'll have to look into this and email Francois, since he wrote the stuff
>>> 20 years ago. Should be no big deal.
>>>
>>> I think that this may take a few days ...
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Apr 11, 2014 at 7:54 PM, Kevin Gullikson <
>>> kevin.gullikson at gmail.com> wrote:
>>>
>>>> Leo,
>>>>
>>>> As far as implementing a writer in Astropy, the trickiest part would be
>>>> writing the Fortran-style format strings. Probably the most reliable method
>>>> would be to write the table data using the existing fixed-width formatter,
>>>> then read it back to deduce the format string.
>>>>
>>>> I have used the fortranformat<https://pypi.python.org/pypi/fortranformat>package for fortran-format strings in the past. I've found that the string
>>>> formatting in python has a really hard time getting everything right for
>>>> input to fortran programs.
>>>>
>>>>
>>>> Kevin Gullikson
>>>>
>>>>
>>>> On Fri, Apr 11, 2014 at 1:34 PM, Leo Singer <lsinger at caltech.edu>wrote:
>>>>
>>>>> Hi Paul,
>>>>>
>>>>> I think that I need to write both the ASCII table and the ReadMe
>>>>> header section. I am preparing some Machine Readable Tables for inclusion
>>>>> in an an ApJ submission, following the guidelines here:
>>>>> https://aas.org/authors/online-only-materials-guidelines
>>>>>
>>>>> I wrote a little script to send an astropy.table.Table instance to the
>>>>> AAS machine readable table converter form. That form is here:
>>>>> http://authortools.aas.org/MRT/upload.html
>>>>>
>>>>> And the script is here:
>>>>> https://gist.github.com/lpsinger/10489886
>>>>>
>>>>> As far as implementing a writer in Astropy, the trickiest part would
>>>>> be writing the Fortran-style format strings. Probably the most reliable
>>>>> method would be to write the table data using the existing fixed-width
>>>>> formatter, then read it back to deduce the format string.
>>>>>
>>>>> Leo
>>>>>
>>>>> On Apr 10, 2014, at 4:28 PM, Paul Kuin <npkuin at gmail.com> wrote:
>>>>>
>>>>> Do you mean that you need to generate the ascii tables as well as the
>>>>> format tables for the ReadMe file?
>>>>>
>>>>> On Thu, Apr 10, 2014 at 11:21 PM, Leo Singer <lsinger at caltech.edu>wrote:
>>>>>
>>>>>>
>>>>>> Does anyone know of a tool for writing Machine Readable Tables (CDS
>>>>>> format) in Python? I know that astropy.table and asciitable can read them,
>>>>>> but I need to write one.
>>>>>>
>>>>>
>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
>


-- 

* * * * * * * * http://www.mssl.ucl.ac.uk/~npmk/ * * * *
Dr. N.P.M. Kuin      (n.kuin at ucl.ac.uk)
phone +44-(0)1483 (prefix) -204927 (work)
mobile +44(0)7806985366  skype ID: npkuin
Mullard Space Science Laboratory  - University College London  -
Holmbury St Mary - Dorking - Surrey RH5 6NT-  U.K.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20140411/f6f9217a/attachment.html>


More information about the AstroPy mailing list