[Tutor] log parser speed optimization [files are iterable]
Alex
python-tutor-list@tagancha.org
Fri May 16 18:25:02 2003
* Tom Jenkins <tjenkins@devis.com> [2003-05-16 18:02:22 -0400]:
> Danny Yoo wrote:
> |
> | I wouldn't be so sure about the slowness here. I think the code here, as
> | it stands, should perform ok. Intuition about performance is always iffy:
> | we really do need to see the results of a 'profile' before trying to
> | prematurely optimize every corner of the log parser.
> |
> i agree wholeheartedly. rereading what i wrote, realized i didn't say
> what i thought i said...
> "There are 2 spots I identified as potential bottlenecks..." could be
> read as "after running through profile i identified 2 spots as potential
> bottlenecks..."
>
> i didn't run anything through the profiler. What i should have said was...
> "There are 2 spots the I'm suspicious could be potential bottlenecks..."
>
> | preferred way to go line-by-line through a file. 'xreadlines', on the
> | other hand, is about to be deprecated, so we should avoid using it:
>
> ack
I apologize, but when "gzip" module is used to open a compressed file I
can't just iterate over it. I get an error about iterating over a
non-sequence. Hence I have to stick to the original "readline()" method
I used, as I understand it because gzipped file is not iterable.
Thank you,
Alex.
--
The lyf so short, the craft so long to lerne.
-- Chaucer
- Life's not fair, but the root password helps.
-- BOFH