[Tutor] newbie confused about text parsing

Chris Johnson chrisj@liquid.sec-d.net
Sun Jul 13 14:45:03 2003


hi group,
    I'm a unix administrator and I want to learn python to help in my job. I
thought parsing a log file would be a good start but I'm stuck on how to
start.

I'm working with a firewall log file the contents of which look something
like this.
  Nov 30 00:58:05 firewall kernel: Shorewall:man1918:DROP:IN=eth0 OUT=
MAC=ff:ff:ff:ff:ff:ff:00:90:f5:1e:15:aa:08:00 SRC=10.1.2.27 DST=10.1.2.255
LEN=96 TOS=0x00 PREC=0x00 TTL=128 ID=4853 PROTO=UDP SPT=137 DPT=137 LEN=76

I'd like to eventually have these put into a mysql data base. But I'm having
trouble figuring out how to parse this.
some of the fields won't be there every time and the file contains other
logs (ie. this came out of /var/log/messages)

I want to loop through log file looking for a string (Shorewall) then parse
each matching line into a dictionary which I can sort or put into database
fields.

I've been reading the documentation on both modules re and string but which
do I use. I'd like to run this script all the time so entries are added in
near real time to the database.

I've actually worked out the MySQLdb part of it pretty well it's just the
text parsing I'm having trouble starting.

sorry if this is post is to log winded.
thank for any suggestions.
chrisj