[Mailman-Users] Mailman throughput

[Brad Knowles]

On 08/14/2011 03:39 PM, Ivan Fetch wrote:

>> There are some MTA tuning tips in the FAQ
>> <http://wiki.list.org/x/AgA3>, but some are only applicable to Mailman
>> 2.0 so be careful.

The majority of the MTA tuning tips that I know of should be applicable 
to most any mailing list manager, since they are oriented towards 
helping the MTA better deal with large amounts of outgoing mail, and 
optimizing certain types of behaviors that are common with most mailing 
lists.

But I'll have to re-fresh my memory of what is written there.

>> The main outgoing MTA performance killer is doing DNS verification on
>> recipient domains during SMTP from Mailman. This should be avoided.
>
> Using a local DNS cache cut my 5000 messages to a 25 recipient list,
> from 10 minutes down to 8& 1/2 minutes. Even avoiding looking up the
> same hand full of hosts over and over again, helps.

Generally speaking, if there are any real-time queries being done by 
your MTA, you want those done against the message as it comes into your 
mail system the first time -- this includes checking black lists, 
checking content, or anything else.

You want to run a separate instance of your MTA for handling your 
outbound mail and it should listen only to a special port on the 
127.0.0.1 "loopback" interface where Mailman can speak directly to it, 
and that special instance should have pretty much all DNS queries and 
real-time checks turned off.  After all, those things should have been 
done when the message was checked on inbound and shouldn't need to be 
checked again on outbound.

> I have to amend my earlier statement about our receiving 68000 posts
> per day - I was not careful enough when mining the post log; a lot of
> the posts are Mailman retrying delivery for tempfailed subscribers. So
> we do not see 68000 distinct posts, but we are doing a lot of redelivery
> attempts. Apparently we need to tune bounce processing for lists - this
> can be challenging to get right, and seems to require individual
> attention per list. I suppose I could have Mailman retry delivery less
> often, and if we have something like an outage of our own relays, I just
> trigger a retry by restarting the queue runners.

If Mailman is dealing with tempfails, then you've done something wrong. 
  The MTA should be blindly accepting whatever Mailman has to send, and 
then the MTA should be dealing with tempfails -- it's one step closer to 
wherever the problem might be, and it's more likely to be tuned for that 
kind of behaviour.

For example, most modern MTAs give you the ability to set up separate 
queues for given outbound targets, which are kept apart from all the 
other regular mail being handled.  This way you can set up "local" 
queues in your MTA that may have different resource handling rules or 
different retry algorithms, as compared to queues to external sites that 
might be known for being troublesome.

We were doing this kind of thing at AOL back in the mid-90s, and this 
has only gotten easier since.

-- 
Brad Knowles <brad at shub-internet.org>
LinkedIn Profile: <http://tinyurl.com/y8kpxu>