File iteration in 2.2

[Erik Max Francis]

Clarence Gardner wrote:

> Here's a question which is mostly curiosity, because I figure there
> *must* be a good reason for this.

This seems to confuse a fair number of people; file iterators are really
intended work like the F.xreadlines() method, which is an optimized form
of F.readlines() method to read in chunks at a time (as you've
discovered).

I think the answer you'd get is that the .readlines/.xreadlines are
really intended when you want to iterate over the _entire_ file no
matter what.  .readlines reads _all_ the lines in at once, not giving
you the opportunity to stop in the middle; .xreadlines is intended to be
an optimization for this, so it doesn't literally have to read
everything into memory at once, but it's intended for the same activity
-- where you're really planning on iterating over every line in the
file.

Consider the behavior if you had done:

	for line in F.readlines():
	    break

This would read in all the lines, advancing F to its end of file, and
then wouldn't do anything with it, breaking out of the loop.  Since for
line in F.xreadlines() and for line in F are intended to do the same
thing, you shouldn't really expect any different behavior; the fact that
the position of F in its file is left in some arbitrary state after is
simply a byproduct of that optimization.

That is, if you want to iterate over the _whole_ file, use for line in
F.xreadlines() or just for line in F.  If you want to read line by line
and stop at some point, use the

	while 1:
	    line = F.readline()
	    if not line:
	        break
	    ...

gimmick.

-- 
 Erik Max Francis / max at alcyone.com / http://www.alcyone.com/max/
 __ San Jose, CA, US / 37 20 N 121 53 W / ICQ16063900 / &tSftDotIotE
/  \ There is nothing so subject to the inconstancy of fortune as war.
\__/ Miguel de Cervantes
    Church / http://www.alcyone.com/pyos/church/
 A lambda calculus explorer in Python.