for line in file weirdness

[Fredrik Lundh]

"Cordula's Web" <cpghost at cordula.ws> wrote:

> here's a strange bug (?) I've came across (using Python 2.2):
>
> # loop_1
> for line in file:
>    if some_condition(line): break
>    do_something()
>
> # loop_2
> for line in file:
>    do_something_else()
>
> The problem is, that loop_2 doesn't resume where loop_1 left off, but
> skips many lines (a block's worth or so) before continuing.
>
> Why is this? Is reading from a file non-reentrant?

as mentioned in the documentation, the iterator interface (which is used by the
for-in machiner) uses a read-ahead buffer.  in 2.2, whenever you enter a new
loop, a new read-ahead buffer is created, and it starts where the last one ended,
rather than right after the last line you read.

to get more reliable results in 2.2, you can create the iterator outside the loop,
and loop over the iterator object instead of the file itself.

    file = iter(open(...))
    for line in file:
        if some_condition(line): break
        do_something()
    for line in file:
        do_something_else()

(iirc, this quirk was fixed in 2.3)

</F>