Lazy file.readlines()?

[Darrell]

Ran the test.py and enabled the whole file test as well.
Looks like readline() is a smoker, but read() needs help.

On NT with a 54meg file and average line length of 147

compare readlines()
Using native readlines
         3 function calls in 10.539 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    1.237    1.237   10.479   10.479 <string>:1(?)
        0    0.000             0.000          profile:0(profiler)
        1    0.060    0.060   10.539   10.539 profile:0(testlines(filespec))
        1    9.243    9.243    9.243    9.243 test.py:30(testlines)


Using QIO readlines
         3 function calls in 10.212 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    1.556    1.556   10.211   10.211 <string>:1(?)
        0    0.000             0.000          profile:0(profiler)
        1    0.001    0.001   10.212   10.212
profile:0(testqiolines(filespec))
        1    8.655    8.655    8.655    8.655 test.py:38(testqiolines)


compare readline() loops
Using native readline
         3 function calls in 31.633 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000   31.632   31.632 <string>:1(?)
        0    0.000             0.000          profile:0(profiler)
        1    0.001    0.001   31.633   31.633 profile:0(test(filespec))
        1   31.632   31.632   31.632   31.632 test.py:5(test)


Using QIO readline
         3 function calls in 5.539 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    5.538    5.538 <string>:1(?)
        0    0.000             0.000          profile:0(profiler)
        1    0.001    0.001    5.539    5.539 profile:0(testqio(filespec))
        1    5.538    5.538    5.538    5.538 test.py:20(testqio)


compare read() of a whole file
standard
Size: 53992540
         2 function calls in 2.427 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    2.425    2.425    2.425    2.425 <string>:1(?)
        0    0.000             0.000          profile:0(profiler)
        1    0.001    0.001    2.427    2.427 profile:0(z = f.read(); print
len(z))


qio
Size: 53992540
         2 function calls in 15.612 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1   15.550   15.550   15.550   15.550 <string>:1(?)
        0    0.000             0.000          profile:0(profiler)
        1    0.061    0.061   15.612   15.612 profile:0(z = f.read(); print
len(z))


--
--Darrell