[Mailman-Users] Mailman 2.1.6 slowness...?

[Brad Knowles]

At 8:45 AM -0400 2005-07-16, Jeff Squyres wrote:

>  Ah -- thanks for clarifying.  This is not really what is implied in the
>  FAQ (http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.012.htp)
>  -- I read it to  imply that mailman is grouping messages to the MTA by
>  domain in order to boost minimum number of deliveries to a given target
>  domain.

	No.  He's talking about the network bandwidth that will be used 
by the MTA, once it has accepted all the messages and recipients from 
Mailman.  The MTA is forced to send no more than X recipients to a 
given target site, because no more than X recipients exist at that 
site.

	But Mailman could dump hundreds of thousands of recipients on the 
local MTA, and not care about what the local MTA does with that. 
We've been talking about the Mailman-to-MTA interface, not the 
MTA-to-MTA interface.

>            That could simply be my flawed interpretation (I based that
>  assumption on the bandwidth discussion in the middle), but perhaps it's
>  worth a few clarifying statements in the FAQ...?

	I'll see if I can come up with something.

>>  	If you break that up into smaller chunks, the system can be processing
>>  more of those chunks in parallel, and each chunk can be accepted faster.
>
>  We have pumped our SMTP_MAX_RCPTS down to 5 (we had never changed it
>  before -- the mailman default was 500, even though the FAQ suggests
>  that it should be significantly lower), but we'll have to wait for our
>  next peak traffic period (likely not until Monday) to see how well
>  it's actually working out.

	One thing I would encourage you to do is to change just one thing 
at a time, and see what the effects are.  With regards to reducing 
SMTP_MAX_RCPTS, I would encourage you to reduce the value by roughly 
half at each stage.  So, go from 500 to 250, 250 to 125, 125 to 62, 
62 to 32, etc....  This way, you should get a better idea of what the 
real threshold is.

>  Right -- sorry, I didn't mean to imply otherwise.  We were not surprised
>  by this, either.  I was trying to say that we've seen this behavior for
>  a long time and didn't have any performance issues with it.

	This sort of thing happens all the time with all sorts of 
systems.  People will notice that their tires seem a little low, and 
there is some smoking coming out of the tailpipe, but they won't do 
anything about it until the car blows up or the tires come off the 
rims, etc....  That's when they take the car to the mechanic.

	With computers, people may notice that queues get really long, 
but they'll think that this is perfectly normal and acceptable, until 
something bad happens.  That's when they go looking for help. 
They've been seeing all the signs that something bad was likely to 
happen soon, but they didn't recognize them for what they were.

	Sometimes it's expensive to fix catastrophic failures, sometimes 
not.  Unfortunately, this is the way that the world tends to work.

>  The thought occurs to me that perhaps it wasn't our sendmail guys who
>  changed something, but perhaps the guys in the anti-spam/virus-checking
>  crew changed something (I believe they also check outgoing mails for
>  some insundry list of things that they believe indicates spam/viruses).
>  Hmm.  Need to go ping them, too...

	Yeah, gotta talk to them, too.  The recommended practice for 
mailing lists is to check messages on input, but don't try to check 
them on output -- after all, the messages were already demonstrated 
to be clean on input.

	You may or may not be able to do this at your site, but you 
should at least check with them.


	The problem could also be DNS or reverse DNS.  Those kinds of 
things can really slow down MTAs, as they check their incoming 
connections.  If a DNS server is flaking out, the MTAs could be 
taking much longer than they used to in order to do all the same 
sorts of checks that they've always been doing.


	Another thing to watch out for is tar-pitting SMTP services, such 
as TarProxy (see <http://www.martiansoftware.com/tarproxy/>) or 
OpenBSD's "spamd" (see 
<http://www.openbsd.org/cgi-bin/man.cgi?query=spamd> and 
<http://www.benzedrine.cx/relaydb.html>).  Note that "spamd" can also 
be run on modern versions of FreeBSD that have also implemented the 
"pf" firewall package that is also taken from OpenBSD.

	This wouldn't directly affect Mailman, but could cause messages 
to become backlogged on the MTA, waiting to be sent.  You want to 
keep a close eye on those MTAs, and you may want to make sure that 
any tar-pitting is only done on inbound mail as opposed to outbound.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

   SAGE member since 1995.  See <http://www.sage.org/> for more info.