[Mailman-Users] Slow delivery

[AE Somerville]

I am currently experiencing a very slow delivery of email from mailman to
our smart relay server. The job of mailman in our system is to accept the
list message expand it and inject it directly back to our mail router for it
to decide where the messages are to go. At no time will the mailman server
connect and deliver a message to the external SMTP host.

 

Oct 06 18:24:02 2004 (21646) <mailman.59.1097038133.9270.acidres at qut.edu.au>
smtp for 1 recips, completed in 9.089 seconds

Oct 06 18:24:30 2004 (21646) <bfxvzepvpvzfjuuvmtk at qut.edu.au> smtp for 1
recips, completed in 27.876 seconds

Oct 06 18:24:40 2004 (21646) <tnvrvmfxofptwtcnywt at qut.edu.au> smtp for 3
recips, completed in 10.496 seconds

Oct 06 18:24:56 2004 (21646) <ttxnisxjtswzmzebxnf at qut.edu.au> smtp for 1
recips, completed in 14.809 seconds

Oct 06 18:25:39 2004 (21646) <vjnpqeylvztebmlpdzp at student.qut.edu.au> smtp
for 96 recips, completed in 41.099 seconds

Oct 06 18:25:44 2004 (21648)
<5.1.1.5.2.20041006144107.029c4e50 at pop.qut.edu.au> smtp for 849 recips,
completed in 584.837 seconds

 

The emails with one recipient are the killers as there are a lot of them and
it they all take in excess of 10sec each then the rest of the list
processing falls behind as a result. It seems like the outbound qrunner
process is taking things at a very slow pace. The installation is on a local
hard drive but the logs are on an NFS mount. All the queue runners are
loading OK and I'm not seeing much by way of errors, its just slow to get
the mail out (2Hrs+) and the machine CPU/RAM is not being used much at all,
generally it's 75% idle. Messages do flow through, its just we get more than
we deliver out so the out directory is overflowing. (when they are flowing
the delivery times to fluctuate wildly; 1 message recipient can take
6.661sec while the 3 recipient message above it in the log only took
0.370sec)

 

We have a lot of messages that reject but we found that the qrunner was
retrying the 100s of them every hour so we lengthened that to every 24Hrs.
(We know the messages are likely to be deleted if they are not accepted in
the first instance by the smtp smart relay, but we need to keep them to
retry in just case of an aliases error on the relay) Other qrunner configs
from the URL included

 

Everything is vanilla except for the site configs below and one code change:

In "mailman-2.1.5/Mailman/Queue/Switchboard.py", change line 167 from

        if not lower or (lower <= long(digest, 16) < upper):

to

        if (lower == upper) or (lower <= long(digest, 16) < upper):

 

 

Settings from mm_cfg.pycat        

#http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq06.006.htp

QRUNNER_LOCK_LIFETIME = hours(20)

QRUNNER_PROCESS_LIFETIME = hours(2) 

QRUNNER_MAX_MESSAGES = 50000

 

 

SMTPHOST = 'smtp.qut.edu.au'

SMTPPORT = 25

 

MAILMAN_OWNER     = 'list-manager-owner@%s' % DEFAULT_HOST_NAME

 

DEFAULT_MAX_MESSAGE_SIZE = 5000           # KB

DEFAULT_SEND_REMINDERS = 0

DEFAULT_SEND_WELCOME_MSG = 0

DEFAULT_MINIMUM_REMOVAL_DATE = 30

DEFAULT_MINIMUM_POST_COUNT_BEFORE_BOUNCE_ACTION = 10

SMTP_MAX_RCPTS = 1000

SMTP_MAX_SESSIONS_PER_CONNECTION = 0 

MAX_DELIVERY_THREADS = 0

DEFAULT_ARCHIVE = Off

DELIVERY_RETRY_PERIOD = days(5)

DELIVERY_RETRY_WAIT = hours(24)