[MATRIX-SIG] Data import into Numeric arrays

[Kevin Rodgers]

I have the need to process large amounts of time history data with NumPy.
The data are stored as ordinary ASCII files with each line being one
timepoint and its associated data.  Currently, the only ways I know to get
this data into NumPy  all involve reading the data into a Python list (say,
via readline or readlines), splitting each line into fields, converting the
string representations of numbers to actual numbers, then using the array()
function to build an array.  This seems really inefficient.  What I'd like
to do is just read the data directly into NumPy from disk as a Numeric
array!  Similarly with writing arrays; it would be really nice to easily
write this kind of data out to disk.  Any ideas on this?  Currently, about
80% of my scripts processing time is in the reading and writing of the
datafiles.  Thanks in advance . . .

(BTW, if you're only suggestion is for me to store the data in a different
format, thanks but no thanks; this data has to be used by lots of different
people, and I am pretty much stuck with this format, as much as I'd like to
move it to, say netCDF or HDF or even straight binary.)
----------------------------------------------------------------------
Kevin Rodgers     Teledyne Ryan Aeronautical  krodgers@tdyryan.com
"This one goes up to eleven."  --  Nigel Tufnel
----------------------------------------------------------------------

_______________
MATRIX-SIG  - SIG on Matrix Math for Python

send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________