[Mailman-Users] Options for increasing throughput
fcocquyt at stanford.edu
Tue Jun 24 02:55:54 CEST 2008
Mike, many thanks for your (as always) very helpful response - I added the 1
liner to mm_cfg.py to increase the threads to 16.
Now I am observing (via memory trend graphs) an acceleration of what looks
like a memory leak - maybe from python - currently at 2.4
I am compiling the latest 2.5.2 to see if that helps - for now the
workaround is to restart mailman occasionally.
(and yes the spamassassin checks are the source of the 4-10 second delay -
now those happen in parallel x16 - so no spikes in the backlog...)
On 6/20/08 9:01 AM, "Mark Sapiro" <mark at msapiro.net> wrote:
> Fletcher Cocquyt wrote:
>> Hi, I am observing periods of qfiles/in backlogs in the 400-600 message
>> count range that take 1-2hours to clear with the standard Mailman 2.1.9 +
>> Spamassassin (the vette log shows these messages process in an avg of ~10
>> seconds each)
> Is Spamassassin invoked from Mailman or from the MTA before Mailman? If
> this plain Mailman, 10 seconds is a hugely long time to process a
> single post through IncomingRunner.
> If you have some Spamassassin interface like
> that calls spamd from a Mailman handler, you might consider moving
> Spamassassin ahead of Mailman and using something like
> or just header_filter_rules instead.
>> Is there an easy way to parallelize what looks like a single serialized
>> Mailman queue?
>> I see some posts re: multi-slice but nothing definitive
> See the section of Defaults.py headed with
> # Qrunner defaults
> In order to run multiple, parallel IncomingRunner processes, you can
> either copy the entire QRUNNERS definition from Defaults.py to
> and change
> ('IncomingRunner', 1), # posts from the outside world
> ('IncomingRunner', 4), # posts from the outside world
> which says run 4 IncomingRunner processes, or you can just add
> something like
> QRUNNERS[QRUNNERS.index(('IncomingRunner',1))] = ('IncomingRunner',4)
> to mm_cfg.py. You can use any power of two for the number.
>> I would also like the option of working this into an overall loadbalancing
>> scheme where I have multiple smtp nodes behind an F5 loadbalancer and the
>> nodes share an NFS backend...
> The following search will return some information.
Senior Systems Administrator
Information Resources and Technology (IRT)
Stanford University School of Medicine
Email: fcocquyt at stanford.edu
Phone: (650) 724-7485
More information about the Mailman-Users