[SciPy-user] Null values in csv

[Martin Blom]

Hello,

I'm trying to import a huge comma (tab really) separated value file into
numpy/scipy. Trouble is, it's a mix of numerical values and 'null' values
(encoded as the string 'null'). I guess I could just import it as an array
with dtype=string32. However, the file is quite big and I'd like to use as
little memory as possible, and worse, it seems like a really ugly solution.
Since dealing with null values in experimental science must be a fairly
standard problem, I was wondering what people in general do when confronted
with them? Is there some standard data type that I have overlooked and
should use? Are there any clever workarounds? Or am I stuck with strings?

The file (which contains DNA microarray data, in case anyone wondered) looks
sort of like this, but bigger:
0.021    -0.041    0.282    0.021    null    0.299
0.198    0.144    null    -0.046    null    null
-0.081    -0.322    null    -0.005    null    null

Thank you
Martin Blom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20070514/0c7c660c/attachment.html>