[Mailman-Users] Performance issues with 12K subscribers and 14K subscribers and personalized footers.

[Mark Sapiro]

On 09/27/2013 10:40 PM, Bryan Wright wrote:
> 
> So back to the speed.   If multiple qrunners won't fix the issue, is
> there any way to speed this up? To get it to process multiple outgoing
> messages at once? 


Before trying to send from Mailman in parallel, you need to look into
speeding up the SMTP interaction between Mailman and sendmail, bevcause
that's where the real delay exists.


> Could I create multiple lists and then just have the lists subscribed to
> the one list?  Then each of these lists could use a qrunner?


Yes and no. You could create an umbrella list or use
regular_include_lists (see the FAQ at <http://wiki.list.org/x/TIA9>) to
create multiple outgoing messages with fewer recipients, but there is no
guarantee that these message's out/ queue entries would all wind up in
different OutgoingRunner slices. Actually ensuring that each message is
processed by a different OutgoingRunner would, I think, require some
kludgey code mods. It would be easier and more productive to modify
SMTPDirect to use multiple threads. (There is an 'experimental'
MAX_DELIVERY_THREADS setting documented in Defaults.py, but it is not
currently implemented.)


> Could disk access time be an issue?  Could I place the mailman queue
> directory in a ram disk? 


Putting the queue on a ram disk won't help. The queue entry is only
picked up at the start of processing and whether it takes microseconds
or milliseconds, it is not the source of any significant delay.

Putting the server's swap space on a ram disk could help, but there are
other things to try first.

As I said in my original reply, see the FAQs at
<http://wiki.list.org/x/doA9> and <http://wiki.list.org/x/q4A9>. A local
caching DNS may help and if sendmail is doing DNS verification at
incoming SMTP time, stopping that will really help. Also ensure you
haven't changed the default

SMTP_MAX_SESSIONS_PER_CONNECTION = 0

setting.

Note: as a point of reference, my production server is not very high
powered, runs CentOS 5, and has no specific performance tuning. Messages
from Mailman use Mailman's VERP feature so each recipient gets an
individual message whether or not personalization is on. Mailman
delivers over 50 messages per second to Postfix or 12,000 messages in 4
minutes. This is based on up to 600+ messages (my maximum list size is
currently 629), and may slow down some with larger lists as Postfix gets
busier sending, but I'm sure delivery to Postfix of 12,000 messages
would take minutes as opposed to hours.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan