steve at pearwood.info
Tue Mar 16 11:24:43 CET 2010
On Tue, 16 Mar 2010 11:27:13 am you wrote:
> I think he thinks that python is going to read the whole 3.6GB of
> data into memory in one hit, rather than using a small amount of
> memory to process it line by line. But "for line in datafile" in your
> code above uses a generator, right? So I don't think it's a problem -
> correct me if I'm wrong.
No, you are correct -- "for line in file" reads one line at a time.
Beware, though, if the file isn't line-oriented, then each "line"
(separated with a newline character) could be huge.
More information about the Tutor