[Numpy-discussion] Memory usage of numpy-arrays

[Christopher Barker]

Sebastian Haase wrote:
> This might be better done "by hand", i.e. in standard (non numpy) python:
> 
> nums = []
> for line in file("myTextFile.txt"):
>      fields = line.split()
>      nums.extend (map(float, fields))

if you know how big your array needs to be, you can pre-allocate it with:

np.empty() or np.ones()

Then fill it in as you read the file -- that will be as memory efficient 
as you can do it.

Another option:

I wrote a expendable array class in Python a while back, which turns out 
not to be faster than using a list, but it should be more memory 
efficient -- you might try it (enclosed).

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Accumulator.zip
Type: application/zip
Size: 4701 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20100708/b3fe5558/attachment.zip>