[Mailman-Users] Big Lists [Postfix]

[Dan Wilder]

I'd like to emphasize that YMMV.  As Chuq notes, don't take my numbers or
anybody else's and apply them blindly.  What fits one site will fit
badly on others.  Change one thing at a time, in some orderly way that
lets you evaluate impact of changes.

Important things I can think of immediately that will vary by site include:

-- Bandwidth to internet

-- Size of email postings

-- Number of postings

-- Number of recipients

-- Number of recipient domains

-- Ease and reliability of reaching recipient domains and their DNS

-- Amount of RAM, processer speed, disk speed in mailserver

-- Other load the mailserver may be under

-- How fast the list mail needs to be delivered

-- Proximity of nearest DNS cache

All of these may impact the optimum tuning.  

On Tue, Dec 18, 2001 at 10:39:39AM -0800, Chuq Von Rospach wrote:
> On 12/18/01 5:06 AM, "Tass" <tass at kenderhome.com> wrote:
> 
> >> default_process_limit = 150
> > 
> > If you have 512M of Ram set it to 200, it will give you a lot of room.
> 
> Maybe. Maybe not. 
> 
> One of the things you need to do when setting up your MTA is figure out what
> your network can take. It makes no sense (in fact, it hurts throughput
> significantly) if you have more processes attempting to send mail than your
> network can handle.
> 
> If you're on a 384K DSL, you have to tune much different than if you're on
> an T1. And you're gated by your slowest link, not your closest one.
> 
> One of the things you need to do, then, is try increasingly large numbers of
> parallel delivery processes, and watch your network patterns. What I usually
> do is look for when dropped packets and/or retries start going up. Once you
> get there, you've basically saturated your slowest network link, bceause
> stuff is disappearing into the void. Once that happens, your overall
> throughput will start going down, because your processes will be fighting
> for the network, losing packets, retrying, waiting for each other's packets
> to get out of the way, and generally creating minor chaos at the TCP layer.
> 
> So what I do is I ramp up delivery until this point happens, then back it
> off by about 10%. If (as with the SSH guys a couple of days ago) you want to
> reserve more bandwidth, back it off more.
> 
> A really rough way to estimate this, if you don't want to get nerdy, is to
> use "ping". Ping, for instance, the router on the far side of your WAN link,
> and wait for packets to start dropping. That'll tell you when that link
> fills up. 
> 
> If you're seeing significant collisions, retries or dropped packets, you're
> overfilling your network and hurting performance. Better 50 mail processes
> cooperating than 60 getting in each other's way. And I think it's best to
> manage this by rate-limiting the MTA, which you can test and tune fairly
> easily.
> 
> >> disable_dns_lookups = yes
> > 
> > I have found this to be no real change.
> > On a PIII-500 with 512M I am getting 100K-120K/hr on a T1 connection with
> > avg message of 6.1K.
> 
> Then I'd guess you have a fairly small group of domains you deal with. The
> wider your subscribers wander, especially if you cross continents, the more
> you'll see issues with slow DNS. Better to turn it off and let the Mta deal
> with it asynchronously.
> 
> > If most of your email is going to a few domains that you know are good at
> > handling email you can also set
> > 
> > default_destination_concurrency_limit = 15
> 
> As long as you don't flood them and piss them off. Be careful over-ramping
> to a single domain.
> 
> 

Yes.  I've got

  default_destination_concurrency_limit = 5

and also

  smtp_destination_recipient_limit = 10

in postfix/main.cf

because of piss-off problems with a few recipient domains.  These don't 
seem to affect our overall delivery rate much, because no one domain accounts 
for more than a few percent of total subscribers.

-- 
-----------------------------------------------------------------
 Dan Wilder <dan at ssc.com>   Technical Manager & Editor
 SSC, Inc. P.O. Box 55549   Phone:  206-782-8808
 Seattle, WA  98155-0549    URL http://embedded.linuxjournal.com/
-----------------------------------------------------------------