[AstroPy] Changing dtype when copying masked table throws TypeError

Aldcroft, Thomas aldcroft at head.cfa.harvard.edu
Fri Aug 21 10:41:22 EDT 2015


On Tue, Jun 16, 2015 at 6:07 PM, Gregory Simonian <
simonian.7 at buckeyemail.osu.edu> wrote:

> Hi all,
>
> I have a Table with several columns which were read in as strings because
> they had a combination of numeric and non-numeric characters. After
> cleaning out the non-numeric characters, I'd like to convert the dtype of
> the column from string to int.
>
> Looking at the astropy documentation for modifying tables [0], under the
> caveats section it states:
>
> "The data type for a column cannot be changed in place. In order to do
> this you must make a copy of the table with the column type changed
> appropriately."
>
> So I try making a new table (the table is small, so I don't mind the
> temporary memory hit), but I run into the error:
> "TypeError: Cannot set fill value of string with array of dtype int64"
> When checking the fill value of the column (which is masked from the
> reader), it's 'N/A', which makes sense that it wouldn't work. But when I
> change it to '0', I still get the same problem. I managed to find a minimal
> working example as:
>
> import numpy as np
> t = Table({'a': ['1']}, masked=True)
> t['a'].set_fill_value('0')
> u = Table(t, names=['a'], dtype=[np.int])
> > TypeError: Cannot set fill value of string with array of dtype int64
>
> When debugging the exception, it seems like the Table constructor doesn't
> convert the fill value of masked tables when generating a new table, which
> seems to defeat the ability to change the dtype of masked columns. I was
> wondering if other people have run by this, and if there's a more
> straightforward way of changing the dtype of a column. This is with Astropy
> 1.0.3 on Python 2.7.10 through Anaconda on CentOS 6.6.
>
> When checking to see if this was reported, I ran into issue #3809 [1],
> which might end up being the ultimate solution to my problem. It seems like
> a workaround would be:
>
> temp_column = np.ma.asanyarray(t['a'], np.int)
> del(t['a'])
> t['a'] = temp_column
>
>
Hi Gregory,

Sorry, your nice bug report fell through the cracks and didn't get a
response.  You've basically understood the issues correctly, and I just put
in a pull request that fixes this:

https://github.com/astropy/astropy/pull/4088/files

In your workaround you could have done something slightly simpler:

 temp_column = t['a'].astype(np.int)

Cheers,
Tom




> Thanks!
>
>                                             Gregory Simonian
>
> [0] http://docs.astropy.org/en/stable/table/modify_table.html#caveats
> [1] https://github.com/astropy/astropy/issues/3809
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150821/80cce2b0/attachment.html>


More information about the AstroPy mailing list