[AstroPy] saving tables as VOtables with missing values

[Thomas Robitaille]

Hi Susana,

Just to simplify your current workflow, what you are doing is equivalent to:

tab = Table(rows=list, names=...)
tab.write('test.xml', format='votable')

so no need for the zip(*) call and the call to votable.from_table.

Now in terms of the masked values, I think the easiest is actually to
just give some integer value to the missing value and then set the mask, so:

In [19]: t = Table(rows=list,
                   names=["A", "B", "C", "D", "E"],
                   masked=True)

In [20]: t['D'].mask[0] = True

In [21]: print(t)
 A    B   C   D   E
---- --- --- --- ---
CIG1 1.2 3.0  -- 3.0
CIG1 3.0 2.0 5.0 2.0
CIG3 2.0 3.0 2.6 nan

and then write out to votable with:

t.write('test.xml', format='votable')

which should preserve the masks.

If you chose to use a special value (e.g. np.nan) to indicate masked
values, then you can do:

t['D'].mask = np.isnan(t['D'].mask)

Let me know if any of the above isn't clear, or if it doesn't solve your
issue!

Cheers,
Tom

Susana Sanchez Exposito wrote:
> 
> Hi all,
> 
> I work on an interface where the user can view and edit tables and save
> them as VOtables, and for that I use the Astropy library.
> 
> I keep the data of the tables in python lists, so to save them as
> VOtable I do this:
> 
> list = [["CIG1", 1.2, 3, "", 3], ["CIG1", 3, 2.0, 5, 2], ["CIG3", 2,
> 3,2.6,  float('nan')]]
> 
> #transform the list of rows into list of columns
> list_cols= zip(*list)
> 
> tab=astropy.table.Table(list_cols, names=["A", "B", "C", "D", "E"])
> votable=astropy.io.votable.from_table(tab)
> votable.to_xml("/home/sse/Desktop/test.xml")
> 
> The table.Table method interprets correctly  the type of each column
> except for the 3rd column. This column contains a "missing value" or
> maybe a value deleted by the user, so the whole columns is marked as
> "string" type, when actually is float type.
> 
> I could transform all empty strings into NaN but this will be strange
> for those columns containing strings .
> 
> I have tried to transform the python list into a masked array, but
> without success: problems to mask empty strings.
> 
> Maybe I should to find out the type of each columns, going over the
> table and calculating the type of the majority of the column item, and
> then pass this type array to the table.Table method in some way ??
> 
> So before, to continue investigating, I would like to ask you for some
> tips to solve the missing values problems with astropy, or maybe even if
> there is a specific method for that.
> 
> Thanks in advanced.
> 
> Susana.
> 
> 
> 
> 
> 
> 
> 
> -- 
> Susana Sánchez Expósito
> 
> Instituto de Astrofísica de Andalucía   IAA (CSIC)
> Camino Bajo de Huétor, 50. Granada E-18008
> Tel:(+34) 958 121 311 / (+34) 958 230 618
> Fax:(+34) 958 814 530
> e-mail: sse at iaa.es <mailto:sse at iaa.es>
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy