[Mailman-Users] Writing a custom handler

[Chris Nulk]

On 7/2/2013 10:16 AM, Mark Sapiro wrote:
> On 07/02/2013 08:48 AM, Chris Nulk wrote:
>
>
> You could change your code as follows:
>
> 1) Make the banlist global by putting
>
> # First, initialize the banlist
> banlist = []
>
> ahead of
>
> def process(mlist, msg, msgdata):
>
> and remove that from the process() definition:
>
> Then make the code inside process() that loads the banlist conditional
> on banlist being the empty list, e.g.,
>
>      if not banlist:
>          ...
>
> Then once the global banlist is loaded with the file data, the file
> won't be read again.
>
>
>> What I would like to do is read the global ban list file once to build
>> the ban list but update the ban list if there has been a change in the
>> global ban list file.
>
> You have a couple of choices:
>
> 1) Do the above and just restart Mailman if you update the banlist.
>
> 2) add code to test and remember (in a global) the mod time of the
> banlist file and read it only if the file is newer.
>
> Method 2) may involve comparable overhead to just reading the file each
> time so may not be worth it. I would either do 1) or just read the file
> every time based on how often I expect the file to change.
>
> Note that if posts are infrequent, the overhead of reading the file each
> time doesn't much matter, and if posts are frequent, the file will
> probably be in an OS disk cache anyway.
>
Mark,

Thanks again for all the help.  I choose method 2.  I have a few 
processes that monitor the log files.  I can modify them to 
automatically update the ban list file based on whatever parameters I 
want to set.  With method 2, my Mailman instance is updated on the next 
message that comes in without me having to do anything.  Our site has 
some activity but not a lot as some places do so I am not worried about 
the overhead.

I did forget about some of my other questions.  I plan on writing 
another custom handler for a list-specific issue.  Where would I look if 
I wanted to intercept messages related to subscribing, unsubscribing, 
and options processing?  How would I tell the difference between a 
subscribe message and an unsubscribe message? Also, is there a 
difference if the person does the subscribing or unsubscribing via the 
web?  Can I trap those?

Below is the latest update incorporating your suggestions.

Thanks again,
Chris

------------------------ Updated Global Ban Custom Handler 
----------------------

#!/usr/bin/env python
#

"""This is a custom handler that will check all the sender addresses of
a message against a global ban list.  If any of the sender addresses are
on the global ban list, the message will get logged and discarded.
"""

import sys
import os

from Mailman import mm_cfg
from Mailman import Utils
from Mailman import Message
from Mailman import Errors
from Mailman.i18n import _
from Mailman.Logging.Syslog import syslog

# Global variables
#   Initialize the banlist
banlist = []

#   Keep the Global Ban lists modification time
#     set to -1 to indicate ban file hasn't been read
ban_mtime = -1

# Define ban_file
#   mm_cfg.GLOBALBANLIST_FILENAME is defined in mm_cfg and should
#   be the full path to the file.
ban_file = mm_cfg.GLOBALBANLIST_FILENAME

def process(mlist, msg, msgdata):
     # Upstream pipeline handler marked message approved -
     #   respect the decision
     if msgdata.get('approved'):
         return

     # ban_file gets its value from mm_cfg.GLOBALBANLIST_FILENAME. If
     #   mm_cfg.GLOBALBANLIST_FILENAME is not defined neither is
     #   ban_file, so simply return.
     if not ban_file:
         return

     # Read in the global ban list of email addresses
     if Ban_File_Changed(ban_file, ban_mlist):
         # Global Ban list has changed (or ban_mlist = -1),
         #   read in the changes
         rc = Read_GlobalBan_File(ban_file)
         if not rc:
             # Problems reading the GlobalBan list
             return

     # Check if banlist has anything, if not, no need to go further
     if not banlist:
         return

     # Go through possible senders.  Check if any of them are
     #   on the global ban list
     for sender in msg.get_senders():
         if sender.lower() in banlist:
             break
     else:
         # None of the sender addresses were in the global ban
         #   list so return and continue with the next pipeline
         #   handler
         return

     # A sender was found on the global ban list.  Log it and
     #   discard the message notifying the list owner
     if sender:
         # Log banned sender to the vette log
         syslog('vette', '%s is banned by the global ban list', sender)
         # Perform message discard
         do_discard_globalban(mlist, msg, sender)
     else:
         assert 0, 'Bad sender in GlobalBan.py'


# Stat the ban file to get the modification time and compare it to the
#   last time the file was changed.  If a changed occured, update
#   ban_mtime to current change time
def Ban_File_Changed(ban_file, ban_mtime):
     try:
         statinfo = os.stat(ban_file)
     except IOError, e:
         # cannot stat the global ban list for whatever reason
         # log it and continue with the next pipeline handler
         syslog('error',
                "Can't stat %s: %s" % (ban_file, e)
                )
         return False
     except:
         # unspecified error
         # log it and continue with the next pipeline handler
         syslog('error',
                'ERROR: %s: %s' % (sys.exc_info()[0], sys.exc_info()[1])
               )
         return False

     # if ban_mtime = -1, statinfo.st_mtime should always be greater, this
     #   is a special case for when the code is first loaded and run
     if statinfo.st_mtime > ban_mtime:
         ban_mtime = statinfo.st_mtime
         return True

     # no change in ban file
     return False


# Read the Global Ban file and populate the banlist.
def Read_GlobalBan_File(ban_file):
     try:
         with open(ban_file) as f:
             for addr in f:
                 # if addr is not in banlist, add it - to avoid duplicates
                 if addr not in banlist:
                     banlist.append(addr.lower().strip())
     except IOError, e:
         # cannot open the global ban list for whatever reason
         # log it and continue with the next pipeline handler
         syslog('error',
                "Can't open %s: %s" % (ban_file, e)
                )
         return False
     except:
         # unspecified error
         # log it and continue with the next pipeline handler
         syslog('error',
                'ERROR: %s: %s' % (sys.exc_info()[0], sys.exc_info()[1])
               )
         return False

     # success
     return True


# copied almost verbatim from Mailman/Handlers/Moderate.py
def do_discard_globalban(mlist, msg, sender):
     # forward auto-discards to list owners?
     if mlist.forward_auto_discards:
         lang = mlist.preferred_language
         nmsg = Message.UserNotification(mlist.GetOwnerEmail(),
                                         mlist.GetBouncesEmail(),
                                         _('Global Ban List Auto-discard 
notification'),
                                         lang=lang)
         nmsg.set_type('multipart/mixed')
         text = MIMEText(Utils.wrap(_("""\
The sender - %(sender)s - of the attached message is on the Global Ban 
list.  Therefore, the message
has been automatically discarded.""")),
                         _charset=Utils.GetCharSet(lang))
         nmsg.attach(text)
         nmsg.attach(MIMEMessage(msg))
         nmsg.send(mlist)
     # Discard the message
     raise Errors.DiscardMessage