.readline() - VERY SLOW compared to PERL

[Harald Schneider]

> Just for interest, I wondered what difference, if any, does it make to the
> Python scripts you posted if you put everything inside a function so that
> it uses local variables in place of the global variables?

No special intention. It just was a quick hack of a beginner :)


> I tried this and, although the times fluctuate substantially on different
> runs, I got your 5.6 second script running on my test file (48Mb) in 12.66
> seconds. My version with local variables took 10.12 seconds.

With my 24 MB file the difference is approx. 0.1 sec. on my machine.
Thanks for that hint!

> Not a major difference, but possibly worthwhile. Interestingly, most of
the
> speedup comes from using a local variable for string.split. Without this
> optimisation it takes about 12.06 seconds. Also note that I used the -O
> command line option as otherwise the times were all about 5 seconds
slower.
>
> Here is my version:
>
> ======================================
> import sys, string, time
>
> def run():
>     print "Running..."
>     dbname = 'test.dat'
>     secStart = time.time()
>
>     db = open(dbname, 'r')
>
>     read = db.readlines
>     split = string.split
>
>     while 1:
>         lines = read(250000)
>         if not lines:
>             break
>         for dbline in lines:
>             rs = split(dbline, ';')
>
>             if rs[0] == 'TEST':
>                 print dbline + "\n"
>                 break
>
>     print "DONE!\n"
>     db.close
>     print "Elapsed time: %f sec." % (time.time() - secStart)
>
> if __name__=='__main__':
>     run()
> ========================================