Modifying mailman to filter archived messages
Sent this to Mailman-users earlier, but am now thinking it is more appropriate here...
All, I'm trying to modify mailman to simplify some tasks. Currently, sysadmins get various emails from LogWatch every day or so that need to be read through in depth and then saved to some mailbox somewhere. This both takes a lot of time and creates lots of private copies of these messages, which is bad. The simple part of this is to instead have the emails go to a mailman list that these people may or may not choose to subscribe to, so that mailman will archive the messages and let people choose whether or not to receive them without the intervention of others. The tough part is status coding. I've already modified LogWatch (and will modify other email-sending scripts such as our backup scripts) to code certain lines based on whether or not they require attention. Example 0: means "fine" or "green" and 2: means "trouble" or "red". What I'd like mailman to do is trap these strings such as "0:" "1:" etc and replace them with something else. Is there a particularl easy way to do this? If not, where in the code could I conceivably do this. I've been briefly skimming the code, and intend to read through quite a bit of it to figure all this out, but if anyone could at least point me in the right direction it'd make me very happy.
Thanks for your help, Laurence Berland
Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
"LB" == Laurence Berland <laurence@digitalpulp.com> writes:
LB> What I'd like mailman to do is trap these strings such as "0:"
LB> "1:" etc and replace them with something else. Is there a
LB> particularl easy way to do this? If not, where in the code
LB> could I conceivably do this. I've been briefly skimming the
LB> code, and intend to read through quite a bit of it to figure
LB> all this out, but if anyone could at least point me in the
LB> right direction it'd make me very happy.A general approach would be to hack into the Mailman/Queue/ArchRunner.py file, ArchRunner._dispose() method. I'd write a separate "handler module", say Mailman/Handlers/OurStatusMunger.py which does the specifics of the transformation you're interested in. Although your situation is fairly unique, you might want to copy the API and style of other handler modules in that directory.
Then add something to _dispose() that checks an attribute on the mlist object to decide whether to call into your handler module. I'd do the check like this:
...
from Mailman.Handlers import OurStatusMunger
...
if getattr(mlist, 'munge_status_p', 0):
OurStatusMunger.process(mlist, msg, msgdata)
...and then use bin/withlist to add this attribute (set to 1 of course!) to just the list you want to do the extra processing on.
Note that if you want to do this /before/ the message hits the archiver, e.g. you want it in the outgoing messages, you'd simply need to add the OurStatusMunger module to the GLOBAL_PIPELINE variable in Defaults.py.in -- another good reason to make OurStatusMunger.py conform to the handler API.
-Barry
On Wednesday 17 July 2002 04:41 pm, Barry A. Warsaw wrote:
"LB" == Laurence Berland <laurence@digitalpulp.com> writes:
... from Mailman.Handlers import OurStatusMunger ... if getattr(mlist, 'munge_status_p', 0):OurStatusMunger.process(mlist, msg, msgdata) ...
I'm mostly following this, but am having a little trouble at the point where I write the process function. Looking at other handlers I've determined I can get a header by msg.gtheader(header-name-string) and get the body by msg.body. I've also determined you can alter headers by setting msg[header-name-string] but what I haven't figured out it what I can do to set the body to something. The idea is to apply a regular expression on the body of the message.
Thanks again for all the help, Laurence
"LB" == Laurence Berland <laurence@digitalpulp.com> writes:
>> ... from Mailman.Handlers import OurStatusMunger ... if
>> getattr(mlist, 'munge_status_p', 0):
>> OurStatusMunger.process(mlist, msg, msgdata)
>> ...
>>
LB> I'm mostly following this, but am having a little trouble at
LB> the point where I write the process function. Looking at
LB> other handlers I've determined I can get a header by
LB> msg.gtheader(header-name-string) and get the body by msg.body.
LB> I've also determined you can alter headers by setting
LB> msg[header-name-string] but what I haven't figured out it what
LB> I can do to set the body to something. The idea is to apply a
LB> regular expression on the body of the message.Ouch, if you're using MM2.0.x then it gets pretty painful to do these kinds of message manipulation. Any chance you can be coaxed into using MM2.1?
There, you've got the email package at your disposal, so modifying message headers and payloads (email package parlance for message bodies), is much more regular and predictable.
E.g. you'd do msg.get(fieldname) to get the contents of a header, and "del msg[fieldname] ; msg[fieldname] = value" to replace the contents of a header. msg.attach(attachment) or msg.set_payload(body) to manipulate the body of a message, etc.
The Python 2.2 documentation is helpful for the basics of the email package, however it has undergone significant extension and API improvements. The documentation has not yet caught up, so you should use the source for the most up-to-date information.
-Barry
On Wednesday 17 July 2002 04:41 pm, Barry A. Warsaw wrote:
A general approach would be to hack into the Mailman/Queue/ArchRunner.py file, ArchRunner._dispose() method. I'd write a separate "handler module", say Mailman/Handlers/OurStatusMunger.py which does the specifics of the transformation you're interested in. Although your situation is fairly unique, you might want to copy the API and style of other handler modules in that directory.
I'm using mailman 2.0.11 and can't seem to find the file ArchRunner.py, or even a Queue directory for that matter. Is this because my version is too old/new? Where would I find this in 2.0.11, or should I just upgrade and port the work I've already done over? If so, what version should I upgrade to, keeping in mind this code needs to actually be used in production in the near future, so beta (2.1?) isn't necessarily a good idea.
Then add something to _dispose() that checks an attribute on the mlist object to decide whether to call into your handler module. I'd do the check like this:
... from Mailman.Handlers import OurStatusMunger ... if getattr(mlist, 'munge_status_p', 0):OurStatusMunger.process(mlist, msg, msgdata) ...
and then use bin/withlist to add this attribute (set to 1 of course!) to just the list you want to do the extra processing on.
I've added a few config flags instead. When I finish with all this, is there any interest in adding it back in to the mailman source? It offers three configurable parameters for archives. Regex filtering on/off, regex pattern, replacement pattern. I can think of a few interesting uses offhand, namely variable spam armoring, automatic filtering of obscenity (if it's a list for children or something, I don't want to claim to generally advocate that sort of thing), gratuitous substitution of phrases (I know some people who'd love to always replace "Microsoft" with "the evil whose name one dares not speak" in every message...), etc.
Note that if you want to do this /before/ the message hits the archiver, e.g. you want it in the outgoing messages, you'd simply need to add the OurStatusMunger module to the GLOBAL_PIPELINE variable in Defaults.py.in -- another good reason to make OurStatusMunger.py conform to the handler API.
I'm trying to conform, and hopefully someone will be willing to glance over stuff when I'm done and tell me if I have conformed sufficiently, and if not what changes I need make.
Thanks again for the extensive help, Laurence
"LB" == Laurence Berland <laurence@digitalpulp.com> writes:
LB> I'm using mailman 2.0.11 and can't seem to find the file
LB> ArchRunner.py, or even a Queue directory for that matter. Is
LB> this because my version is too old/new?Too old. MM2.0.x is in the critical patch only phase of its life, and even that is getting difficult to do correctly (witness the 2.0.12 problem w/ Python 1.5.2).
LB> Where would I find this in 2.0.11, or should I just upgrade
LB> and port the work I've already done over? If so, what version
LB> should I upgrade to, keeping in mind this code needs to
LB> actually be used in production in the near future, so beta
LB> (2.1?) isn't necessarily a good idea.Depends. MM2.1 is probably the only one that has the architecture you'll need to do the things I think you'll want to do. It's mostly stable enough.
LB> I've added a few config flags instead. When I finish with all
LB> this, is there any interest in adding it back in to the
LB> mailman source? It offers three configurable parameters for
LB> archives. Regex filtering on/off, regex pattern, replacement
LB> pattern. I can think of a few interesting uses offhand,
LB> namely variable spam armoring, automatic filtering of
LB> obscenity (if it's a list for children or something, I don't
LB> want to claim to generally advocate that sort of thing),
LB> gratuitous substitution of phrases (I know some people who'd
LB> love to always replace "Microsoft" with "the evil whose name
LB> one dares not speak" in every message...), etc.Possibly, but MM2.1 is pretty well feature frozen.
LB> I'm trying to conform, and hopefully someone will be willing
LB> to glance over stuff when I'm done and tell me if I have
LB> conformed sufficiently, and if not what changes I need make.The best approach, IMO is to post your work as a patch to the SF patch managers, and announce it here so folks can take a look.
-Barry
On Wednesday 24 July 2002 00:52, Barry A. Warsaw wrote:
Too old. MM2.0.x is in the critical patch only phase of its life, and even that is getting difficult to do correctly (witness the 2.0.12 problem w/ Python 1.5.2).
Has there been an official release of 2.1?
"PB" == Phil Barnett <philb@philb.us> writes:
>> Too old. MM2.0.x is in the critical patch only phase of its
>> life, and even that is getting difficult to do correctly
>> (witness the 2.0.12 problem w/ Python 1.5.2).
PB> Has there been an official release of 2.1?Betas only, still. -Barry
participants (4)
-
barry＠python.org -
barry＠zope.com -
Laurence Berland -
Phil Barnett