Warning about "for line in file:"

[Russell E. Owen]

I tried taking advantage of
  for line in file:
(a recent addition to the language) and ran into some odd behavior.

I read some lines using one for loop, to get to the start of the data of 
interest. E.g.:
for line in file:
  if startingcondition(line):
    break

I then some more lines from the same file using a similar for loop, but 
found that the lines were not contiguous! Instead there was a huge gap 
between the two sets of reads, as if the first for loop read in a lot of 
extra data that it never spit out.

I am guessing this is some form of optimization, but it is very 
surprising behavior. I expected "for line in file:" to act as an 
iterator -- read some now, read some later, whenever you read you get 
the next line.

Anyway, beware of treating file too much as an iterator.

There are plenty of workarounds. In my case the fileinput module 
actually was a better match for my needs (interleaving data from several 
sets of files). The readline or xreadline file methods work fine, of 
course. One could presumably use the iterator returned by 
file.__iter__(), but I didn't try that.

-- Russell
-- 
Return      owen
address     astro
garbled     washington
in header   edu