[Mailman-Users] Script to clean shunt directory

[David Vasil]

Until I can figure out how to stop the UnicodeDecodeErrors
and mail quits getting shunted, I figure it would be best to
keep the shunt directory somewhat clean.  To do this I wrote
a script that checks each .pck in the shunt directory for
the X-Spam-Level header to determine the liklihood of the
message being spam.  If it is highly likely the message is
spam, the script removes the .pck and .db file, otherwise it
warns the person running the script to investigate possible
legit email that ended up in shunt.

Can anyone see anything wrong with removing files in the
shunt directory in this manner?  The script I wrote is
attached.

-- 
+------------------------------------------------------------+
| Dave Vasil                                vasil at cs.utk.edu |
| University of Tennessee             Computer Science Dept. |
| UTKCS Systems Administrator                   865-974-8364 |
+------------------------------------------------------------+
-------------- next part --------------
#!/bin/bash
#
# Written by Dave Vasil <vasil at cs.utk.edu>
# May 5th, 2004
# Distributed under the GPL V2 or higher.
#
# Cleans out the mailman shunt directory in mailman/qfiles/shunt
#
# At the time this was written (May 5th 2004) several emails to
# lists would be shunted because they could not be parsed correctly
# due to illegal charset chars being used.  It appears that these
# messages are primarily caused by spammers
#
# This script tries to differentiate between messages with high
# spam levels and legitimate email.  Liklihood of a legit email being
# held in shunt is low, but if legit mail is held, this script 
# will send notice to the user running the script to investigate 
# low spam scoring messages.
#
# Script requires some sort of content filtering before mailman
# receives the message.  In this case, amavisd-new adds a header
# X-Spam-Level: XXXXX  (where the XXXXX is the level of spam
# liklihood).
#
# Distributed by default with $CLEAN set to false, you must
# explicitly set it to 1 if you wish to remove files.

SHUNTDIR=/usr/local/mail/mailman/qfiles/shunt
MAXSPAMX=7
CLEAN=0
TEST=1
SHUNTFILNUM=`ls $SHUNTDIR/*.pck 2>/dev/null | wc -l`

if [ $SHUNTFILNUM -gt 0 ]; then
  # Just for verbosity
  if [ $TEST -eq 1 ]; then
    echo "Checking $SHUNTFILNUM shunted messages in $SHUNTDIR."
	fi
	# Start the main loop
  for shuntfile in `ls $SHUNTDIR/*.pck`
  do
	  # egrep the shuntfile for the spam level
    spamlevel=`egrep -o -a -e 'X-Spam-Level...X+' $shuntfile | wc -c`
		# subtract 14 from the spam level for X-Spam-Level:
    spamlevel=`expr $spamlevel - 14`
    if [ $TEST -eq 1 ]; then
      echo "Shunted file: `basename $shuntfile` with spam level $spamlevel."
    fi
		# If $CLEAN is set, and $spamlevel > $MAXSPAMX, remove the .pck and .db for that $shuntfile
    if [ $CLEAN -eq 1 ]; then
      if [ $spamlevel -gt $MAXSPAMX ]; then
        if [ $TEST -eq 1 ]; then
          echo "Cleaning `basename $shuntfile` from the qfiles dir since spam exceeds $MAXSPAMX."
        fi
        rm -f $shuntfile "`echo $shuntfile | sed 's/\.pck$/\.db/'`"
      else
			  # Possible legit mail based on $MAXSPAMX
        echo "Please investigate `basename $shuntfile`.  Spam level ($spamlevel) is below $MAXSPAMX.  Possible valid message."
      fi
    fi
  done
else
  # Just for verbosity
  if [ $TEST -eq 1 ]; then
    echo "There are no shunted messages in $SHUNTDIR.  Lucky you."
	fi
fi
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/mailman-users/attachments/20040505/17e21b0b/attachment.pgp