![](https://secure.gravatar.com/avatar/09ea2202a148196130137fefdbbf32c2.jpg?s=120&d=mm&r=g)
I don't think I can do that because I have heterogeneous rows of data.... I.e. the columns in each row are different in length. Furthermore, when reading it back in, I want to read only bytes of the info at a time so I can save memory. In this case, I only want to have one record in mem at once. Another issue has arisen from taking this routine cross-platform.... namely, if I write the file on Windows I cant read it on Solaris. I assume the big-little endian is at hand here. I know using the struct module that I can pack using either one. Perhaps I will have to go back to the drawing board. I actually love these methods now because I get back out directly what I put in. Great kudos to the developers.... MJ -----Original Message----- From: numpy-discussion-bounces@scipy.org [mailto:numpy-discussion-bounces@scipy.org] On Behalf Of Christopher Barker Sent: Tuesday, February 13, 2007 1:39 PM To: Discussion of Numerical Python Subject: Re: [Numpy-discussion] fromstring, tostring slow? Mark Janikas wrote:
I am finding that directly packing numpy arrays into binary using the tostring and fromstring methods
For starters, use fromfile and tofile, to save the overhead of creating an entire extra string. fromfile is a function (as it is an alternate constructor for arrays): numpy.fromfile() ndarray.tofile() is an array method. Enclosed is your test, including a test for tofile(), I needed to make the arrays much larger, and use time.time() rather than time.clock() to get enough time resolution to see anything, though if you really want to be accurate, you need to use the timeit module. My results: Using lists 0.457561016083 Using tostring 0.00922703742981 Using tofile 0.00431108474731 Another note: where is the data coming from -- there may be ways to optimize this whole process if we saw that. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov