Help with script with performance problems
googlegroups at spacerodent.org
Sun Nov 23 19:58:20 CET 2003
Ville Vainio <ville.spammehardvainio at spamtut.fi> wrote in message news:<du7r7zz4ei3.fsf at amadeus.cc.tut.fi>...
> > f = sys.stdin
> Have you tried using a normal file instead of stdin? BTW, you can
> iterate over a file easily by "for line in open("mylog.log"):". ISTR
> it's also more efficient than readline()'s, because it caches the
> lines instead of reading them one by one. You can also get the line
> numbers by doing "for linenum, line in enumerate(open("mylog.log")):"
i have a 240207 line sample log file that I test with. The script I
submitted parsed it in 18 seconds. My perl script parsed it in 4
The new python script, using a normal file as suggested above, does it
in 3 seconds!
Changed "f = sys.stdin" to "f = open('sample', 'r')".
Note (I made the other changes one at a time as well - the file open
change was the only one that made it faster)
More information about the Python-list