iterators and generators, is this the Python Way???
oren-py-l at hishome.net
Thu Sep 12 09:26:46 CEST 2002
On Wed, Sep 11, 2002 at 10:17:05PM -0400, Michael Schneider wrote:
> Hello All.
> I just updated my python to 2.2 and noticed that generators, and
> iterators were in there.
> I often parse through many files (one at a time), and wanted the speed
> of read, and
> the convience of for x in myGen:
"for x in fileobject" uses xreadlines for readahead buffering. It's faster
than reading a file with readline and should be faster than anything you
can implement in Python. If you must have backward compatibility to
Python 2.1 use "for x in fileobject.xreadlines()".
Python 2.3 makes files into their own iterators. It's even faster than
> The code below works just fine, could you review this code against the
> intent of iterators and generators.
IMHO, much of the intent of iterators and generators is to *avoid* reading
everying into memory in advance. This way the memory requirements can often
remain constant regardless of input size. Another good reason not to read
everything into memory is if the source is a real-time stream instead of a
file and you don't even know when it's going to end.
More information about the Python-list