xreadlines (was Re: while true: !!!)

[Greg Jorgensen]

"Jeff Epler" <jepler at inetnebr.com> wrote in message
news:slrn93g37o.9tj.jepler at potty.housenet...
> It seems that the most common place that people write the 'while 1:
> ... if test: berak' idiom is when iterating over lines from a file
> where the file is not known to fit in memory.
> ...
> I take this cue to suggest the idea of an 'xreadlines' object.  I have
> implemented 'xreadlines' in Python (where it was slower than all other
> solutions, probably due to method call overhead) and in C (where it
> holds its own with the fastest idiom I know,

There is the fileinput module, which also solves the "ugly infinite loop"
problem that seems to bother some people:

import fileinput

for line in fileinput.input(filename):
    print line

The fileinput module can iterate over a single file, a list of files, or
stdin. I've used fileinput to go through big lists of files (10,000+ email
messages) and it works great. It doesn't appear to do any buffering
itself--it uses file.readline() to read the files.

--
Greg Jorgensen
Deschooling Society
Portland, Oregon, USA
gregj at pobox.com