[Python-Dev] RE: [Patches] [Patch #102915] xreadlines : readlines :: xrange : range

[Tim Peters]

Final report (I've spent way more time on this than I can afford already, so
it's "final" by defn <0.3 wink>).  We started here (on my Win98SE box, using
Guido's test program):

total 117615824 chars and 3237568 lines
count_chars_lines    14.780 14.772
readlines_sizehint    9.390  9.375
using_fileinput      66.130 66.157
while_readline       30.380 30.337

Here's where we are today:

total 117615824 chars and 3237568 lines
count_chars_lines    14.670 14.667
readlines_sizehint    9.500  9.506
using_fileinput      28.670 28.708
while_readline       13.680 13.676
for_xreadlines        7.630  7.635

Same box, same input file, same test program except for this addition:

def for_xreadlines(fn):
    f = open(fn, MODE)
    for line in xreadlines.xreadlines(f):
        pass
    f.close()

This last is within 25% of Perl "while (<>)" speed, but-- unlike Perl --is
thread-safe.  Good show!  The other speedups are nothing to snort at either.

The strangest thing left to my eye is why xreadlines enjoys a significant
advantage over the double-loop buffering method (readlines_sizehint) on my
box; reducing the very large (1Mb) buffer in Guido's test program made no
material difference to that.

nothing's-ever-finished-but-everything-ends-ly y'rs  - tim