iterating over lines in a file

Tim Peters tim_one at email.msn.com
Sun Jul 23 21:43:43 EDT 2000


[Cliff Crawford]
> No, what I meant was that I thought readlines() always reads the entire
> file, whether you specify a size argument or not, and so if you were
> worried about "for line in file.readlines():" eating too much memory,
> you could just do "for line in file.readlines(8192):" instead, and it
> would read the entire file using only an 8k buffer.  At least, that's
> what I thought the library reference was saying..but the source code
> proved me wrong :)

Ah.  It *is* normally used to read the entire file, but with another level
of loop:

    while 1:
        # read next batch of lines
        lines = file.readlines(8192)  # or larger for more speed
        if not lines:
            break
        for line in lines:
            process(line)

> I do agree, though, that the current behavior of readlines() is quite
> useful for it's >intended< purpose ;)

Well, the "hint" argument is there to let you turn yourself into a compiler
<wink>.






More information about the Python-list mailing list