Mark Janikas wrote:
I don't think I can do that because I have heterogeneous rows of data.... I.e. the columns in each row are different in length.
like I said, show us your whole problem... But you don't have to write.read all the data at once with from/tofile() anyway. Each of your "rows" has to be in a separate array anyway, as numpy arrays don't support "ragged" arrays, but each row can be written with tofile()
Furthermore, when reading it back in, I want to read only bytes of the info at a time so I can save memory. In this case, I only want to have one record in mem at once.
you can make multiple calls to fromfile(), thou you'll have to know how long each record is.
Another issue has arisen from taking this routine cross-platform.... namely, if I write the file on Windows I cant read it on Solaris. I assume the big-little endian is at hand here.
yup.
I know using the struct module that I can pack using either one.
so can numpy. see the "byteswap" method, and you can specify a particular endianess with a datatype when you read with fromfile(): a = N.fromfile(DataFile, dtype=N.dtype("<d"), count=20) reads 20 little-endian doubles from DataFile, regardless of the native endianess of the machine you're on. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov