[Mailman-Users] Large site mailman config
brad at stop.mail-abuse.org
Fri Mar 3 15:43:17 CET 2006
At 12:13 PM +0000 2006-03-03, George Barnett wrote:
> The processes dont seem to be failing. New list mail is delivered fine,
> but what's 'stuck' in the queue doesn't leave until a restart which
> effectively does a queue run.
Ahh. Because Mailman uses the filesystem as a queueing method,
there is a certain FIFO nature to the messages that are processed,
and if the queue is deep then new messages that come in won't be
processed for a while -- the qrunner is going to be at a certain
point in the inode filestructure for the directory and is not going
to look at earlier points in the directory even if they are now free
and would sort earlier in the process.
I believe that there is some discussion on this in FAQ 6.6.
> These were increased after some queues were getting more full than
> others. Changing these values did help, but I suspect it's fixed
> the symptoms rather than the problem.
If you haven't already looked at FAQ 6.6, I'd encourage you to
read it and compare it with your own experience, and the different
values recommended as compared to your own. Using different values
is fine, but you should understand what the values mean and why they
are what they are.
One thing I can tell you is that the filesystem is a critical
bottleneck for both MTAs and MLMs, and most of the performance tuning
techniques for MTAs with regard to filesystems will be equally
applicable to filesystem issues for Mailman and other MLMs.
Towards that end, I'd recommend you take a look at FAQ 6.3 as
well, and the slides at
<http://www.shub-internet.org/brad/papers/sendmail-tuning/> from my
invited talk "Sendmail Performance Tuning for Large Systems" that I
gave at SANE'98. With regards to disk and filesystem performance,
pretty much everywhere it says "Sendmail", you can substitute
"Mailman" instead, without loss of generality.
> I found this information from the faq on python.org. Our traffic pattern
> is very bursty. Lists are small, maybe 25 users per list, but when
> traffic comes in, it's alot at the same time (monitoring mails).
Whereas our traffic tends to be more consistent -- receiving
anywhere from 5000 to 30,000 messages per hour, and around 400,000
total incoming messages per day, and rejecting about 200,000 messages
per day (mostly spam).
By Mailman standards, this is actually only "moderate" size, even
though a couple of months ago we were rejecting 90-95% of all
incoming mail as spam, and handling 500,000-600,000 (or more)
incoming messages per day that were legitimate.
Oh, and we've got some stuff we're doing with the firewall
(blocking known abusive IP addresses for short periods of time),
which doesn't show up in any of our logs. There's enough of that
crap that we can't log that information because the impact of doing
the logging would probably kill the machine.
Brad Knowles, <brad at stop.mail-abuse.org>
"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."
-- Benjamin Franklin (1706-1790), reply of the Pennsylvania
Assembly to the Governor, November 11, 1755
LOPSA member since December 2005. See <http://www.lopsa.org/>.
More information about the Mailman-Users