[Tutor] log parser speed optimization [files are iterable]

[Danny Yoo]

> the del call is unnecessary.  so replace all your pops with the
> appropriate slice.
>
> |         while 1:
> |             line = logfile.readline()
> |             if not line:
> |                 break
>
> classic way to read files.  also slow.  look at xreadlines.



Hi Tom,


I wouldn't be so sure about the slowness here.  I think the code here, as
it stands, should perform ok.  Intuition about performance is always iffy:
we really do need to see the results of a 'profile' before trying to
prematurely optimize every corner of the log parser.


That being said, let's look at that looping code.  *grin* The code above
can be rewritten to take advantage of a file's iterability.  We can write
the top code as:

###
for line in logfile:
    ...
###


This may or may not be faster than the original code, but at least it's
slightly simpler. This iterating approach works in Python 2.2, and is the
preferred way to go line-by-line through a file.  'xreadlines', on the
other hand, is about to be deprecated, so we should avoid using it:

    http://www.python.org/doc/current/lib/module-xreadlines.html



Hope this helps!