speed question, reading csv using takewhile() and dropwhile()

[Jonathan Gardner]

On Fri, Feb 19, 2010 at 1:58 PM, Vincent Davis <vincent at vincentdavis.net>wrote:

> In reference to the several comments about "[x for x in read] is basically
> a copy of the entire list. This isn't necessary." or list(read). I had
> thought I had a problem with having iterators in the takewhile() statement.
> I thought I testes and it didn't work. It seems I was wrong. It clearly
> works. I'll make this change and see if it is any better.
>
> I actually don't plan to read them all in at once, only as needed, but I do
> need the whole file in an array to perform some mathematics on them and
> compare different files. So my interest was in making it faster to open them
> as needed. I guess part of it is that they are about 5mb so I guess it might
> be disk speed in part.nks
>
>

Record your numbers in an array and then work your magic on them later.
Don't store the entire file in memory, though.

-- 
Jonathan Gardner
jgardner at jonathangardner.net
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20100219/0679aedb/attachment.html>