[AstroPy] table string length truncated after reading table

[Matthew Craig]

I ran into this once upon a time too (in atPy).

The type for the columns when you read from a table is being guessed at by ascii; in the absence of any other guidance it assumes the dtype of each column is the length of the longest string it finds in the column.

Doesn't look like there is a way to specify type in ascii.read (though I just did a quick skim of the docs), but this would work:

```
In [35]: thirdTable = table.Table(np.array(secondTable), names=('col1', 'col2'), dtypes=('S50', 'S50'))

In [36]: thirdTable.add_row(('abcdefghijklmnopqrst', 'longer_string'))

In [37]: print thirdTable
        col1              col2
-------------------- -------------
         abcdefghijk  short_string
    abcdefghijklmnop   long_string
abcdefghijklmnopqrst longer_string
```

Matt Craig
PS Would have been happy to answer on astrobabel but waiting for my membership to be approved :)


Office hours/schedule at: http://physics.mnstate.edu/craig 
----
Professor
Department of Physics and Astronomy 
Minnesota State University Moorhead
1104 7th Ave S, Moorhead MN 56563

phone: (218) 477-2439 
fax: (218) 477-2290

On Aug 15, 2013, at 8:45 PM, Josh Walawender <jmwalawender at gmail.com> wrote:

> Hi all,
> 
> I'm having a problem working with astropy.table and astropy.io.ascii and I can't tell if the behavior I'm encountering is a feature or a bug. I'm hoping someone can guide me to a good solution. Here's the situation:
> 
> I have code which loops though a series of input data files, does analysis, stores the results in an astropy.table, and writes the table to a text file using astropy.io.ascii. One of the fields in the row is the input filename. As the code loops through the input files, it reads the previous table as output by io.ascii, appends a row to the table object, then overwrites the old file with a new one based on the new table which contains the new row.
> 
> The symptom is that the all subsequent times through, the length of all strings written to the file name field are now truncated to whatever length the first file name was.
> 
> Here's a quick test case (copied and pasted from iPython) demonstrating the problem:
> 
> ```
> In [1]: import astropy.table as table
> 
> In [2]: import astropy.io.ascii as ascii
> 
> In [4]: firstTable = table.Table(names=('col1', 'col2'), dtypes=('S50', 'S50'))
> 
> In [5]: firstTable.add_row(('abcdefghijk', 'short_string'))
> 
> In [6]: firstTable.add_row(('abcdefghijklmnop', 'long_string'))
> 
> In [7]: print(firstTable)
>       col1           col2
> ---------------- ------------
>      abcdefghijk short_string
> abcdefghijklmnop  long_string
> 
> In [8]: ascii.write(firstTable, "mytable.txt")
> 
> In [9]: secondTable = ascii.read("mytable.txt")
> 
> In [10]: print(secondTable)
>       col1           col2
> ---------------- ------------
>      abcdefghijk short_string
> abcdefghijklmnop  long_string
> 
> In [11]: secondTable.add_row(('abcdefghijklmnopqrst', 'longer_string'))
> 
> In [12]: print(secondTable)
>       col1           col2
> ---------------- ------------
>      abcdefghijk short_string
> abcdefghijklmnop  long_string
> abcdefghijklmnop longer_strin
> ```
> 
> Any suggestions on how to avoid this behavior?
> 
> thanks!
> Josh
> 
> P.S.  Based on the recent discussion on astropy-dev about where to get help, I've also posted this on astrobabel:
> http://www.astrobabel.com/v/discussion/77/astropy-question-table-string-length-truncated-after-reading-table#Item_1
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20130815/aa79a26b/attachment.html>