[Mailman-Users] Mailman throughput

[Mark Sapiro]

On 8/14/2011 1:39 PM, Ivan Fetch wrote:
> 
> On Aug 14, 2011, at 11:39 AM, Mark Sapiro wrote:
>>
>> It seems the major hurdle is in processing the 'out' queue. It is
>> possible to slice OutgoingRunner to provide some parallelism in this
>> process and that may speed things up, but I suspect that a lot of the
>> time is in network communications between OutgoingRunner and the
>> remote Postfix and that slicing OutgoingRunner may not help much, but
>> it would be worth rerunning your benchmark with 2 or 4 outgoing runner
>> slices to see if it helps.
>>
> BY slicing, do you mean setting MAX_DELIVERY_THREADS in mm_cfg.py (restarting Mailman of course)? I did this, with values of 2 and 4, and if it made a difference for my smaller benchmark of 5000 messages, to one list of 25 recipients, it was only seconds of improvement.


No. Threaded delivery in SMTPDirect.py was an experimental feature in
Mailman 2.0. It was never implemented for Mailman 2.1 although the
setting and its documentation were not removed from Defaults.py. Setting
this in mm_cfg.py has no effect. Any difference would be due to random
variation or other factors.

What I meant was to put something like

try:
    QRUNNERS.remove(('OutgoingRunner', 1))
    QRUNNERS.append(('OutgoingRunner', 2))
except ValueError:
    pass

in mm_cfg.py and restart Mailman. The above will cause Mailman to start
two copies of OutgoingRunner with each processing half of the hashed
queue space. See the

#####
# Qrunner defaults
#####

section in Defaults.py for more info.


[...]
> I have to amend my earlier statement about our receiving 68000 posts per day - I was not careful enough when mining the post log; a lot of the posts are Mailman retrying delivery for tempfailed subscribers. So we do not see 68000 distinct posts, but we are doing a lot of redelivery attempts. Apparently we need to tune bounce processing for lists - this can be challenging to get right, and seems to require individual attention per list. I suppose I could have Mailman retry delivery less often, and if we have something like an outage of our own relays, I just trigger a retry by restarting the queue runners.


Just FYI, bounce processing never sees the retries until such time as
Mailman's retry processing gives up on the delivery (default after 5 days).

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan