[Tutor] Re: Speeding up file processing?

[Lee Harr]

#!/usr/bin/python

import re

f = open("today.test") # the server log file

fields = []

for line in f:
    if re.match('^shirky.com', line): # find hits from my site
        fields = line.split()
        try: referer = fields[11] # grab the referer
        except: continue          # continue if there is a mangled line
        referer = re.sub('"', '', referer)
        if re.search("shirky", referer): continue # ignore internal links
        if re.search("-", referer):      continue # ...and email clicks
        referer = re.sub("www.", "", referer)
        print referer



I am not sure, but I think string methods will be faster than using
all those regular expressions:

if line.startswith('shirky')
referer.replace('"', '')
referer.find('shirky') >= 0

Makes the code easier to read (for me) anyhow.

If you try it, let us know if it helps...

_________________________________________________________________
The new MSN 8: advanced junk mail protection and 2 months FREE* 
http://join.msn.com/?page=features/junkmail