Mailman 3 May 2004 - Mailman-Developers

Mailman Size Issues
by Kory Wheatley May 7, 2004

May 7, 2004

I have a critical situation that I need addressed, and it deals with large messages being sent to mailman mailing lists. Currently, we have a lot of large mailing lists (some have 10,000 to 15,000 subscribers) and some of these lists is all Faculty/Staff email subscribers that have an account on our mail server. All messages sent to these lists are moderated. Here's the problem, when some of these pending messages are accepted to be sent, and one of the messages happens to be 5mb in size, that amount of email that is sent to our mail server all at once, "kills us", and someday if a larger one is sent, it will fill up our mail stores and corrupt our mail server. We can configure our mail server to not accept a message over 5 meg and that would solve the problem, but our University Advisory Committee would not allow that. They have approved that it be set to 10 meg, for messages be sent to only one recipient, but individuals who use the mailing lists, abuse this. Sometimes messages get approved that shouldn't and we run into this problem All get to my point, is there a way that if a moderator tries to approve a message that is over 1 meg, that maybe some custom python code could be developed to pop up a message and say "Only messages below 1 meg can be approved" and have the message discarded, or tell the user the only option they have is to discard the message. I know there's the "max_message_size" option you can set, but that still sends the message to the pending database and they can still accept the message, rather than discard the message when its above that size. Any Solution? I know you will want to say change your policy on your mail server, but at this point it's not a option. -- Kory Wheatley Academic Computing Analyst Sr. Phone 282-3874 ######################################### Everything must point to him.

2 1

envelope header checking
by Nadim Shaikli May 7, 2004

May 7, 2004

I have a number of lists that I administer that seem to have come across a nasty quirk. I use postfix and spamassassin to keep spam in check but recently I've noticed something different. I disallow any posts coming in externally with @my_domain via postfix (it uses the envelope sender to do the checks). Mailman, on the other hand, uses the header From: field and thus the problem. I've recently got mails that looks like this, From him(a)outside.com Date: blah blah From: me(a)my_domain.com so postfix processes it and allows it into my queues (after all it is not being spoofed - the envelope (first From) is clean). Mailman though sizes up this email based on the From: header field and denotes it a locally generate post which I allow irrespective of subscription (I have non-member filtering to allow posts from '.*(a)my_domain.com' through via the 'accept_these_nonmembers' setting). So I have 2 questions, 1. Anyone know of a decent way to make sure that both the envelope sender and the noted From header field do in fact match ? else reject/discard the email. 2. Does mailman allow one to check the envelope sender instead of the From: header field ? That way I know both postfix and mailman are looking at the same thing. I'm using postfix-2.0.16; question-1 above also needs to account for myself being able to mail out without issue (ie. in case someone suggests header filtering within postfix -- that would stop internal emails from going out and I'm unable to upgrade to postfix-2.1 just yet which seems to have hooks for this now). Regards, - Nadim __________________________________ Do you Yahoo!? Win a $20,000 Career Makeover at Yahoo! HotJobs http://hotjobs.sweepstakes.yahoo.com/careermakeover

1 0

customize auto-response Subject header? [Mailman-Developers]
by James Ralston May 5, 2004

May 5, 2004

The Subject: header of auto-responses is fixed as: Subject: Auto-response for your message to the "%(listname)" mailing list I'd really like to be able to customize that. E.g.: Subject: thanks for your submission Or, with variable substitution: Subject: thanks for your submission to %(listname) Or even: Subject: Re: %(old_subject_header_contents) I don't think allow list owners to customize the format of the Subject: header of auto-replies is unreasonable. I haven't studied the code closely yet, but I don't think it would be that complicated, either. Thoughts? Regards, James

1 0

Maybe you guys can help me
by Paul Tomblin May 5, 2004

May 5, 2004

I tried asking this on mailman-users, and got totally ignored both times I asked. Here is my problem: One of my mailing list users has an ISP that refuses email for hours at a time, and the ISP that they get their email through, panix.com, sends a warning if it's undelivered for four hours. Problem is, mailman treats that warning as an error, and if it happens for a few days in a row (even if the problem corrects itself for part of the day and delivers the mail), it bounces her off my mailing lists. The problem is that I don't know Python. As far as I can tell, the fix to ignore these warnings should go in Mailman/Bouncers/Postfix.py, but that's all I can figure out. Can somebody help me? Here is a sanitized version of the bounce: ----- Forwarded message from mailman(a)xcski.com ----- Subject: Bounce action notification From: mailman(a)xcski.com To: XXXXXXX-owner(a)xcski.com Date: Sat, 17 Apr 2004 16:36:23 -0400 X-Spam-Status: No, hits=-4.5 required=6.0 tests=AWL,BAYES_00,NO_REAL_NAME autolearn=no version=2.63 This is a Mailman mailing list bounce action notice: List: XXXXXXX Member: XXXXXXX(a)echonyc.com Action: Subscription disabled. Reason: Excessive or fatal bounces. The triggering bounce notice is attached below. Questions? Contact the Mailman site administrator at mailman(a)xcski.com. Date: Sat, 17 Apr 2004 16:36:49 -0400 (EDT) From: MAILER-DAEMON(a)panix.com (Mail Delivery System) Subject: Delayed Mail (still being retried) To: XXXXXXX-bounces(a)xcski.com Content-Description: Notification This is the Postfix program at host l2mail1.panix.com. #################################################################### # THIS IS A WARNING ONLY. YOU DO NOT NEED TO RESEND YOUR MESSAGE. # #################################################################### Your message could not be delivered for 4.0 hours. It will be retried until it is 5.0 days old. For further assistance, please send mail to <postmaster> The Postfix program <XXXXXXX(a)echonyc.com>: connect to echonyc.com[198.67.15.2]: Connection refused Content-Description: Delivery error report Reporting-MTA: dns; l2mail1.panix.com Arrival-Date: Sat, 17 Apr 2004 10:05:49 -0400 (EDT) Final-Recipient: rfc822; XXXXXXX(a)echonyc.com Action: delayed Status: 4.0.0 Diagnostic-Code: X-Postfix; connect to echonyc.com[198.67.15.2]: Connection refused Will-Retry-Until: Thu, 22 Apr 2004 10:05:49 -0400 (EDT) [snip] -- Paul Tomblin <ptomblin(a)xcski.com> http://xcski.com/blogs/pt/ I think I'd like to see a Simpsons episode starting up with Bart Simpson writing 'I will not attempt to undermine the Usenet Cabal'. -- J. D. Falk

5 10

problems with Header.py
by Heiko Rommel May 5, 2004

May 5, 2004

Hi, I experienced this bug in 2.1.14: May 05 12:49:21 2004 (2345) Uncaught runner exception: May 05 12:49:21 2004 (2345) Traceback (most recent call last): File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 110, in _oneloop self._onefile(msg, msgdata) File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 160, in _onefile keepqueued = self._dispose(mlist, msg, msgdata) File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 130, in _dispose more = self._dopipeline(mlist, msg, msgdata, pipeline) File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 153, in _dopipeline sys.modules[modname].process(mlist, msg, msgdata) File "/usr/lib/mailman/Mailman/Handlers/CookHeaders.py", line 74, in process prefix_subject(mlist, msg, msgdata) File "/usr/lib/mailman/Mailman/Handlers/CookHeaders.py", line 237, in prefix_subject headerbits = decode_header(subject) File "/usr/lib/mailman/pythonlib/email/Header.py", line 113, in decode_header raise HeaderParseError HeaderParseError May 05 12:49:21 2004 (2345) SHUNTING: 1083566823.2704201+9f0320cd3fb74d1981b2484dafb3d406c4e3943a For debugging purposes, I attached the shunted message files. -- Heiko Rommel rommel(a)suse.de SUSE Linux AG, Maxfeldstr. 5, D-90409 Nuernberg T: +49 (0) 911 74053 0 F: +49 (0) 911 741 77 55

1 0

problems with Generator.py
by Heiko Rommel May 5, 2004

May 5, 2004

Hi, I experienced this bug with 2.1.14: May 05 12:49:21 2004 (2345) Uncaught runner exception: string payload expected: <type 'list'> May 05 12:49:21 2004 (2345) Traceback (most recent call last): File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 110, in _oneloop self._onefile(msg, msgdata) File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 160, in _onefile keepqueued = self._dispose(mlist, msg, msgdata) File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 130, in _dispose more = self._dopipeline(mlist, msg, msgdata, pipeline) File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 153, in _dopipeline sys.modules[modname].process(mlist, msg, msgdata) File "/usr/lib/mailman/Mailman/Handlers/SpamDetect.py", line 108, in process g.flatten(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 103, in flatten self._write(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 131, in _write self._dispatch(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 157, in _dispatch meth(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 200, in _handle_text raise TypeError, 'string payload expected: %s' % type(payload) TypeError: string payload expected: <type 'list'> May 05 12:49:21 2004 (2345) SHUNTING: 1083538445.4679279+8226bb6aa539e458952456fe2b2a0028198aa560 For debugging purposes, I attached the shunted message files. -- Heiko Rommel rommel(a)suse.de SUSE Linux AG, Maxfeldstr. 5, D-90409 Nuernberg T: +49 (0) 911 74053 0 F: +49 (0) 911 741 77 55

1 0

reporting bugs with sourceforge ?!?
by Heiko Rommel May 5, 2004

May 5, 2004

Hi, hen going to the Bug tracker of Mailman (http://sourceforge.net/tracker/?group_id=103) and clicking on Reporting I get ---------- Error No Valid User Object ---------------- I tried to report some bugs which I will sent to this list instead. -- Heiko Rommel rommel(a)suse.de SUSE Linux AG, Maxfeldstr. 5, D-90409 Nuernberg T: +49 (0) 911 74053 0 F: +49 (0) 911 741 77 55

1 0

critical bug in Digest
by Heiko Rommel May 5, 2004

May 5, 2004

Hi, we experienced some problems with one of our major mailing lists which had the Digest turned on. I finally found out that one of our users had misconfigured his MUA and send messages with Content-Type: text/plain; charset*=utf-8342200235''utf-8%E2%80%9D where Content-Type: text/plain; charset=utf-8 would be correct. This caused Mailman 2.1.14 to a) throw an exception: Apr 30 13:39:51 2004 (13945) Traceback (most recent call last): File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 110, in _oneloop self._onefile(msg, msgdata) File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 160, in _onefile keepqueued = self._dispose(mlist, msg, msgdata) File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 130, in _dispose more = self._dopipeline(mlist, msg, msgdata, pipeline) File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 153, in _dopipeline sys.modules[modname].process(mlist, msg, msgdata) File "/usr/lib/mailman/Mailman/Handlers/ToDigest.py", line 91, in process send_digests(mlist, mboxfp) File "/usr/lib/mailman/Mailman/Handlers/ToDigest.py", line 132, in send_digests send_i18n_digests(mlist, mboxfp) File "/usr/lib/mailman/Mailman/Handlers/ToDigest.py", line 306, in send_i18n_digests msg = scrubber(mlist, msg) File "/usr/lib/mailman/Mailman/Handlers/Scrubber.py", line 182, in process charset = part.get_content_charset(lcset) File "/usr/lib/mailman/pythonlib/email/Message.py", line 800, in get_content_charset charset = unicode(charset[2], pcharset).encode('us-ascii') LookupError: unknown encoding b) store the message in the digest.mbox, but shunt the message (so it was not delivered). I tried the latest patches to Scrubber but that didn't help. To my opinion, this is a very critial error since a single user can shut down the whole mailing list effectively. My only option as an emergency action was to switch off the Digest option and have a look into the very large digest.mbox :( -- Heiko Rommel rommel(a)suse.de SUSE Linux AG, Maxfeldstr. 5, D-90409 Nuernberg T: +49 (0) 911 74053 0 F: +49 (0) 911 741 77 55

1 0

Re: [Mailman-Developers] Maybe you guys can help me
by J C Lawrence May 4, 2004

May 4, 2004

On Tue, 4 May 2004 10:48:16 -0700 Chuq Von Rospach <chuqui(a)plaidworks.com> wrote: > On May 4, 2004, at 10:38 AM, Bob Puff@NLE wrote: > Which ties back to the original problem. Those "warning" > not-really-bounces are anachronisms of the days when all this was tied > to UUCP. Good point. > which basically boils down to "if it looks like a bounce and quacks > like a bounce and swears like a bounce, then dammit, it's a bounce". I'm generally of the mind that my responsibility as list admin is to deliver an SMTP packet stream containing the mail to one of their MXes. Once I've done that my job is done. What they decide to do from there is their problem, and if their choices annoy me it may reflect on my willingness to keep trying to deliver messages to them. TANSTAAFL. Non-conformant bounces and DSNs are annoying choices, intentional or otherwise. At some point this fact needs to be faced. > And the only way to make THEM fix their damned systems to not bounce > stuff that shouldn't bounce is to make it a visible problem to > them. You don't do that by "fixing" mailman. Unfortunately, it'll > cause some pain for users, but my feeling is, if their stuff is > bouncing and they odn't realize it, that's a bigger problem they > SHOULD be warned about,a nyway. cf Usenet death penalty. > so my policy is now simple: if you bounce it back, I'll treat it like > a bounce. If it has the word "spam" in it anywhere, it'll be treated > like a complaint against my server, and suffer immediate > unsubscription. I've a minor tendency to whack entire domains that abuse the abuse@ line. > Tweaking this stuff in mailman is dealing with a symptom, but not > encouraing the real problem to be looked at. Quite. -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

1 0

Re: [Mailman-Developers] Maybe you guys can help me
by J C Lawrence May 4, 2004

May 4, 2004

On Tue, 04 May 2004 11:40:49 -0400 Carson Gaspar <carson(a)taltos.org> wrote: > --On Tuesday, May 04, 2004 10:25 AM -0400 J C Lawrence <claw(a)kanga.nu> > wrote: >> Reliably distinguishing is non-trivial. > Incorrect. Parsing RFC-standard DSNs is trivial. I wrote the code to > do it back in the old days when I ran the firewalls mailing list. That's true only while you maintain the "RFC-standard DSNs" qualifier. Very few of the DSNs my lists receive are RFC-conformant. -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

2 1