[Mailman-Users] Big Lists

[Dan Wilder]

On Mon, Dec 17, 2001 at 05:22:37PM -0800, Bill Moseley wrote:
> At 04:26 PM 12/17/01 -0800, J C Lawrence wrote:
> >On Mon, 17 Dec 2001 15:32:39 -0800 
> >Bill Moseley <moseleymm at hank.org> wrote:
> >
> >> Someone posted a few days or so ago asking about list sizes.  Was
> >> there any response to that query?
> >
> >Yup, and its already in the FAQ.
> 
> http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.015.htp
> 
> Now, anyone available to fill in the blanks for qmail or Postfix tuning?

I'll try to start that ball rolling.

As lists get larger, Postfix delivery with out-of-the box configuration
really slows down.  The bottlenecks I've found are queue length and number 
of SMTP processes, both of which default to values too small for large
lists.  I began noticing pretty severe rate limiting at about 
10,000 deliveries on a list.

To get the number of SMTP processes up, change "default_process_limit"
in main.cf. 

default_process_limit = 150

gave results I could live with.

There's an active message queue which also became a sticky wicket
for us.  Two limits which affect this, and the values I arrived
at by experiment are:

qmgr_message_active_limit = 40000
qmgr_message_recipient_limit = 40000

Default on both of these is 1000, as of the Postfix version I initially
installed.  The comments in the config file say:

# The qmgr_message_active_limit parameter limits the number of
# messages in the active queue.

# The qmgr_message_recipient_limit parameter limits the number of
# in-memory recipients. This parameter also limits the size of the
# short-term, in-memory destination status cache.

With these changes in place, a K6-350 with 128M RAM delivers
messages averaging perhaps 3K over a medium-speed DSL,
without entirely saturating the DSL, at load average below
2.0, without impinging on swap.  It'll reach 20,000
recipients in a couple of hours.

Another common recommendation with Postfix is to set

disable_dns_lookups = yes

a measure others have reported favorably on, but which 
I have not yet tried.

After changing parameters in main.cf, run "postfix reload" then
look at the process table to see if the postfix processes
are running, and how many smtp processes are working at it.  

mailq

gives a report of what's in process and is a great help
in tuning.  A crude index of what's out there is

mailq | grep '^[A-Z0-9]' | wc -l

and variations.


-- 
-----------------------------------------------------------------
 Dan Wilder <dan at ssc.com>   Technical Manager & Editor
 SSC, Inc. P.O. Box 55549   Phone:  206-782-8808
 Seattle, WA  98155-0549    URL http://embedded.linuxjournal.com/
-----------------------------------------------------------------