script to add dates to IIS web log files

[Greg Jorgensen]

I wrote this script last night to add dates to each line of web server log
files if the date is missing. This is apparently a Microsoft IIS feature: if
you don't configure it to add dates to each line it puts the date in a #date
header at the top of the file. The analog web log analyzer doesn't like
this. I downloaded a VBScript script to fix this from the analog web site,
but it could only process one file at a time, so I wrote my own.

I'm still learning Python so this may not be the most elegant
implementation, but it does work and is fast. And I can use it on an entire
directory or set of files specified with wildcards.

Does anyone know if IIS writes another #Date: header into the middle of the
log file if the date changes while the log file is active? If so my script
doesn't account for that. If log file spans two days it won't be detected. I
should probably make the script check for times lower than the previous
time, which would indicate rolling past midnight.

Please send comments/suggestions to me. Thanks.

#!/bin/python
# analogdate -- add date to each line of log files if date is missing
#
# 9 Sep 2000 Gregory Jorgensen, gregj at pobox.com
#
# Note: written for Python 1.6, but easily made compatible with 1.5x by
tweaking string methods
#
# May be freely used and distributed -- please leave my name in the source
file

import sys
import os.path
import glob

# constants
space = ' '

adddate = 0         # true if log file needs dates added
logdate = ""        # date to add

if len(sys.argv) < 2:
        print "usage: python analogdate logfile [logfile...]"
        print "wildcards are expanded, e.g. ex*.log will work"
        print "output files are saved with 'an' prefix"
        sys.exit()

# get command line args
for i in range(1,len(sys.argv)):
    files = glob.glob(sys.argv[i])
    for f in files:
        print os.path.basename(f) + '...'
        try:
            infile = open(f)
            outfile = open(os.path.join( os.path.dirname(f), 'an' +
os.path.basename(f) ), 'w')

            # look at headers to get log date, fix Fields list if found
            while 1:
                s = infile.readline()
                if not s or s[0] != '#':
                    break

                t = s.split()
                if t[0] == '#Date:':
                   logdate = t[1]
                elif t[0] == '#Fields:':
                    if t.count('date') == 0:
                        adddate = 1
                        t.insert(1,'date')
                        s = space.join(t) + '\n'

                outfile.write(s)

            # process the rest of the file, prepending date as necessary
            while s:
                if adddate and logdate and s[0] != '#':
                    s = logdate + ' ' + s
                outfile.write(s)
                s = infile.readline()

            infile.close()
            outfile.close()
        except:
            print '** problem processing ' + f