Looping Moderator Approval Requests / Mailman&Exim
I am having trouble with held posts for the lists I run at Jennic.
We've recently set up Mailman to run in conjunction with Exim and it's working very well (withlist -r is invaluable though), but we're having trouble with the moderation because it is ending up in a loop (that thankfully stops itself after about 30 iterations) but is nonetheless proving to be a big pain.
When a post is received that is held for moderation, an e-mail is sent to list-owner, but when that is sent back into mailman for forwarding, mailman treats the message as if it was going onto the list and holds it (because of the included message that was originally help). This then generates another moderation e-mail which is sent to list-owner which is then held and so on.
I'm not sure if there is a fundamental failure in my understanding of the system of if there's something I've not thought about, but I cannot get it to work properly.
I've seen a lot of documentation suggesting that every list should have 10 aliases in the Exim aliases file for piping into mailman, and -owner is included in these, however I think I am right in saying that that doesn't apply in this situation because I have configured the Exim transport/router as directed to pipe all mail into mailman that hasn't been caught by aliases instead.
Can anyone shed any light on this for me?
P.S. The main issue here is that we run SpamAssassin on the mailserver and it adds an X-Spam_Bar header which we then check for in the list Spam Filters. Before, every single spam e-mail was generating 30 moderator requests.. now we have told the lists to discard anything with an X-BeenThere: ListName@company, but it doesn't just stop the loop, it prevents all moderator requests reaching the list owner because in the end it seems the moderator requests are trying to head for the lists not the list owners. It's all very confuzzling.
Kind Regards,
Mike Peachey, IT Tel: +44 114 281 2655 Fax: +44 114 281 2951 Jennic Ltd, Furnival Street, Sheffield, S1 4QT, UK Comp Reg No: 3191371 - Registered In England http://www.jennic.com
Mike Peachey wrote:
When a post is received that is held for moderation, an e-mail is sent to list-owner, but when that is sent back into mailman for forwarding, mailman treats the message as if it was going onto the list
This is the problem. Exim is piping mail to listname-owner@example.com to "| /path/to/mail/mailman/ post listname" and it should be piped to "| /path/to/mail/mailman/ owner listname".
and holds it (because of the included message that was originally help). This then generates another moderation e-mail which is sent to list-owner which is then held and so on.
I'm not sure if there is a fundamental failure in my understanding of the system of if there's something I've not thought about, but I cannot get it to work properly.
It is an exim configuration problem.
I've seen a lot of documentation suggesting that every list should have 10 aliases in the Exim aliases file for piping into mailman, and -owner is included in these, however I think I am right in saying that that doesn't apply in this situation because I have configured the Exim transport/router as directed to pipe all mail into mailman that hasn't been caught by aliases instead.
Exim is normally configured with a router and transport for Mailman so aliases are not needed, but the router and transport have to be correctly configured.
See http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.019.htp for pointers to exim's documentation on setting this up.
-- Mark Sapiro msapiro@value.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro wrote:
This is the problem. Exim is piping mail to listname-owner@example.com to "| /path/to/mail/mailman/ post listname" and it should be piped to "| /path/to/mail/mailman/ owner listname".
But surely, that would affect a single separate e-mail sent directly to list-owner@ ? If I send to that address, delivery succeeds to the correct recipients.
It is an exim configuration problem.
Still possible, but I'm having trouble seeing how.
Exim is normally configured with a router and transport for Mailman so aliases are not needed, but the router and transport have to be correctly configured. See http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.019.htp for pointers to exim's documentation on setting this up.
I do seem to have it all set up correctly, but feel free to correct me:
<snip> MM_HOME=/usr/lib/mailman MM_UID=mailman MM_GID=mailman domainlist mm_domains=company.com MM_WRAP=MM_HOME/mail/mailman MM_LISTCHK=/var/lib/mailman/lists/${lc::$local_part}/config.pck <snip> mailman_router: driver = accept domains = +mm_domains require_files = MM_LISTCHK local_part_suffix_optional local_part_suffix = -admin : \ -bounces : -bounces+* : \ -confirm : -confirm+* : \ -join : -leave : \ -owner : -request : \ -subscribe : -unsubscribe transport = mailman_transport <snip> mailman_transport: driver = pipe command = MM_WRAP \ '${if def:local_part_suffix \ {${sg{$local_part_suffix}{-(\\w+)(\\+.*)?}{\$1}}} \ {post}}' \ $local_part current_directory = MM_HOME home_directory = MM_HOME user = MM_UID group = MM_GID <snip> -- Kind Regards,
Mike Peachey, IT Tel: +44 114 281 2655 Fax: +44 114 281 2951 Jennic Ltd, Furnival Street, Sheffield, S1 4QT, UK Comp Reg No: 3191371 - Registered In England http://www.jennic.com
Mike Peachey wrote:
Mark Sapiro wrote:
This is the problem. Exim is piping mail to listname-owner@example.com to "| /path/to/mail/mailman/ post listname" and it should be piped to "| /path/to/mail/mailman/ owner listname".
But surely, that would affect a single separate e-mail sent directly to list-owner@ ? If I send to that address, delivery succeeds to the correct recipients.
OK. But then why does your email to listname-owner get properly delivered to the owners, but Mailman's mail to listname-owner get delivered to the list?
It is an exim configuration problem.
Still possible, but I'm having trouble seeing how.
The configuration excerpt you posted below looks correct, but perhaps there is prior router that only applies to mail from localhost that uses an alias or something else and delivers to the list. Is mailman_router the FIRST defined router?
I now notice in your original post you said "I have configured the Exim transport/router as directed to pipe all mail into mailman that hasn't been caught by aliases instead." This suggests mailman_router is not first and some other router which ignores the -owner suffix is routing this mail.
mailman_router should be the first router.
Also, you might check your exim-main-log to see how these messages are routed and delivered.
-- Mark Sapiro msapiro@value.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 7/27/07 8:51 AM, "Mark Sapiro" msapiro@value.net wrote:
mailman_router should be the first router.
Also, you might check your exim-main-log to see how these messages are routed and delivered.
The command line command
exim -bt listname-owner@example.com
may provide some insight. The -bt flag tells exim to explain its routing of the address. For example
$ /usr/sbin/exim -bt test8-owner@lists.olympus.net test8-owner@lists.olympus.net router = mailman_router, transport = mailman_transport
First line shows the routed address; second shows the router and transport. Multiple levels are reported similarly.
--John
Mark Sapiro wrote:
Mike Peachey wrote:
Mark Sapiro wrote:
This is the problem. Exim is piping mail to listname-owner@example.com to "| /path/to/mail/mailman/ post listname" and it should be piped to "| /path/to/mail/mailman/ owner listname".
But surely, that would affect a single separate e-mail sent directly to list-owner@ ? If I send to that address, delivery succeeds to the correct recipients.
OK. But then why does your email to listname-owner get properly delivered to the owners, but Mailman's mail to listname-owner get delivered to the list?
I have discovered that the problem lies with the way SpamAssassin is implemented, but I'm still not sure of the best way around it.
We use SpamAssassin via Exim's spamd_address directive via a unix socket, so every single e-mail that is received by the mail server is run through SpamAssassin before it's routed. We then use the Spam Filters option in Mailman to filter based on the X-Spam_Bar: header.
The problem is that when Mailman sends a moderator request, it attaches the moderated message. Mailman sends this message from it-bounces@ to it-owner@ HOWEVER because the original spam message is attached, SpamAssassin marks mailman's "moderator approval needed" message as spam with the relevant headers.
When this e-mail to it-owner@ is received by Mailman, because it matches the spam filter rules for the it@ list, it holds the message and sends out another moderator request with both previous messages attached.
This then suggests that we have two options: pretty inefficient way to deal with it because everything will still end
- Find a way to stop mailman from attaching the held message to a moderator approval request e-mail
- Find a way to reconfigure Exim & SpamAssassin so that all messages are passed through SpamAssasin, except those that are routed to Mailman, and then use the info available for integrating mailman and SpamAssassin to have Mailman call SpamAssassin itself, although this does seem like a
up going through spamd.
I heartily appreciate any comments on this as I'm a bit stuck with the best way to proceed.
Kind Regards,
Mike Peachey, IT Tel: +44 114 281 2655 Fax: +44 114 281 2951 Jennic Ltd, Furnival Street, Sheffield, S1 4QT, UK Comp Reg No: 3191371 - Registered In England http://www.jennic.com
Mike Peachey wrote:
Mark Sapiro wrote:
Mike Peachey wrote:
Mark Sapiro wrote:
This is the problem. Exim is piping mail to listname-owner@example.com to "| /path/to/mail/mailman/ post listname" and it should be piped to "| /path/to/mail/mailman/ owner listname".
But surely, that would affect a single separate e-mail sent directly to list-owner@ ? If I send to that address, delivery succeeds to the correct recipients.
OK. But then why does your email to listname-owner get properly delivered to the owners, but Mailman's mail to listname-owner get delivered to the list?
I have discovered that the problem lies with the way SpamAssassin is implemented, but I'm still not sure of the best way around it.
We use SpamAssassin via Exim's spamd_address directive via a unix socket, so every single e-mail that is received by the mail server is run through SpamAssassin before it's routed. We then use the Spam Filters option in Mailman to filter based on the X-Spam_Bar: header.
The problem is that when Mailman sends a moderator request, it attaches the moderated message. Mailman sends this message from it-bounces@ to it-owner@ HOWEVER because the original spam message is attached, SpamAssassin marks mailman's "moderator approval needed" message as spam with the relevant headers.
When this e-mail to it-owner@ is received by Mailman, because it matches the spam filter rules for the it@ list, it holds the message and sends out another moderator request with both previous messages attached.
This then suggests that we have two options: pretty inefficient way to deal with it because everything will still end
- Find a way to stop mailman from attaching the held message to a moderator approval request e-mail
- Find a way to reconfigure Exim & SpamAssassin so that all messages are passed through SpamAssasin, except those that are routed to Mailman, and then use the info available for integrating mailman and SpamAssassin to have Mailman call SpamAssassin itself, although this does seem like a
up going through spamd.
I heartily appreciate any comments on this as I'm a bit stuck with the best way to proceed.
Or option 3. find a way to get Mailman to ignore the list Spam Filter settings for the -owner addresses.
Kind Regards,
Mike Peachey, IT Tel: +44 114 281 2655 Fax: +44 114 281 2951 Jennic Ltd, Furnival Street, Sheffield, S1 4QT, UK Comp Reg No: 3191371 - Registered In England http://www.jennic.com
Mike Peachey wrote:
Mike Peachey wrote:
This then suggests that we have two options:
- Find a way to stop mailman from attaching the held message to a moderator approval request e-mail
This is on the ToDo list for Mailman 2.1.10. In the mean time see http://sourceforge.net/tracker/index.php?func=detail&aid=760567&group_id=103&atid=300103 for a patch.
- Find a way to reconfigure Exim & SpamAssassin so that all messages are passed through SpamAssasin, except those that are routed to Mailman, and then use the info available for integrating mailman and SpamAssassin to have Mailman call SpamAssassin itself, although this does seem like a pretty inefficient way to deal with it because everything will still end up going through spamd.
I heartily appreciate any comments on this as I'm a bit stuck with the best way to proceed.
Or option 3. find a way to get Mailman to ignore the list Spam Filter settings for the -owner addresses.
put these four lines (lines 1 and 3 at the left margin, lines 2 and 4 indented 4 spaces) in mm_cfg.py.
try: del OWNER_PIPELINE[OWNER_PIPELINE.index('SpamDetect')] except ValueError: pass
This will remove SpamDetect from the pipeline of handlers through which messages to list-owner are processed, thus skiping the processing of header_filter_rules and mm_cfg.KNOWN_SPAMMERS for mail to the list-owner addresses.
-- Mark Sapiro msapiro@value.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Outbound e-mail from Mailman should always bypass any spam filtering
stages -- those things should have been done on input and not be
necessary to perform a second time.
-- Brad Knowles brad@shub-Internet.org
Sent from my iPhone
On Jul 31, 2007, at 6:06 AM, Mike Peachey mike.peachey@jennic.com
wrote:
Mark Sapiro wrote:
Mike Peachey wrote:
Mark Sapiro wrote:
This is the problem. Exim is piping mail to listname-owner@example.com to "| /path/to/mail/mailman/ post listname" and it should be
piped to "| /path/to/mail/mailman/ owner listname".But surely, that would affect a single separate e-mail sent
directly to list-owner@ ? If I send to that address, delivery succeeds to the correct recipients.OK. But then why does your email to listname-owner get properly delivered to the owners, but Mailman's mail to listname-owner get delivered to the list?
I have discovered that the problem lies with the way SpamAssassin is implemented, but I'm still not sure of the best way around it.
We use SpamAssassin via Exim's spamd_address directive via a unix socket, so every single e-mail that is received by the mail server is run through SpamAssassin before it's routed. We then use the Spam Filters option in Mailman to filter based on the X-Spam_Bar: header.
The problem is that when Mailman sends a moderator request, it
attaches the moderated message. Mailman sends this message from it-bounces@ to it-owner@ HOWEVER because the original spam message is attached, SpamAssassin marks mailman's "moderator approval needed" message as
spam with the relevant headers.When this e-mail to it-owner@ is received by Mailman, because it
matches the spam filter rules for the it@ list, it holds the message and sends out another moderator request with both previous messages attached.This then suggests that we have two options: like a
- Find a way to stop mailman from attaching the held message to a moderator approval request e-mail
- Find a way to reconfigure Exim & SpamAssassin so that all messages are passed through SpamAssasin, except those that are routed to
Mailman, and then use the info available for integrating mailman and
SpamAssassin to have Mailman call SpamAssassin itself, although this does seempretty inefficient way to deal with it because everything will still
end up going through spamd.I heartily appreciate any comments on this as I'm a bit stuck with the best way to proceed.
Kind Regards,
Mike Peachey, IT Tel: +44 114 281 2655 Fax: +44 114 281 2951 Jennic Ltd, Furnival Street, Sheffield, S1 4QT, UK Comp Reg No: 3191371 - Registered In England http://www.jennic.com
Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/brad%40shub-internet.or...
Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp
Brad Knowles wrote:
Outbound e-mail from Mailman should always bypass any spam filtering stages -- those things should have been done on input and not be necessary to perform a second time.
Outbound messages are not filtered. Messages to it-owner@ from it-bounces@ are sent from Mailman to Mailman. They pass through filtering on the inbound connection back into Mailman.
Kind Regards,
Mike Peachey, IT Tel: +44 114 281 2655 Fax: +44 114 281 2951 Jennic Ltd, Furnival Street, Sheffield, S1 4QT, UK Comp Reg No: 3191371 - Registered In England http://www.jennic.com
Sorry, my assumption was that the outbound service could also directly
deliver the mail locally, thus skipping the additional "inbound" phase.
My fault.
-- Brad Knowles brad@shub-Internet.org
Sent from my iPhone
On Jul 31, 2007, at 9:16 AM, Mike Peachey mike.peachey@jennic.com
wrote:
Brad Knowles wrote:
Outbound e-mail from Mailman should always bypass any spam
filtering stages -- those things should have been done on input and
not be necessary to perform a second time.Outbound messages are not filtered. Messages to it-owner@ from it- bounces@ are sent from Mailman to Mailman. They pass through
filtering on the inbound connection back into Mailman.Kind Regards,
Mike Peachey, IT Tel: +44 114 281 2655 Fax: +44 114 281 2951 Jennic Ltd, Furnival Street, Sheffield, S1 4QT, UK Comp Reg No: 3191371 - Registered In England http://www.jennic.com
participants (4)
-
Brad Knowles -
John W. Baxter -
Mark Sapiro -
Mike Peachey