[Mailman-Users] Mailman / Sendmail horribly slow performance.. suggestions please?!?

[Richard Barrett]

On Monday, August 25, 2003, at 04:59  am, Andrew Falgout wrote:

> Sendmail usually delivers regular mail on an almost instant basis.   
> What
> I'm getting it a feel that qrunner is not passing the mails from the
> qfiles directory for sendmail to deliver.
If the qrunner is executing then it is trying to do just that. Can you  
see a qrunner instance when you run ps?
> The machine in question has
> around 15k uses on it, and they have almost instant mail delivery  
> through
> sendmail for regular mail.  When a mail comes to the list through  
> sendmail
> initially, it gets delivered through the mqueue for sendmail and is  
> posted
> to the qfiles of mm almost instantly.  However that is where the holdup
> is.  Qrunner doesn't seem to be dealing with these files well at all.
Or it could be that the MTA is not accepting outgoing mail from MM with  
the same alacrity with which it delivers it.
>
> They are sitting in the qfiles directory and are being delivered or  
> sent
> out at a rate of 1-2 per minute.
>
> I don't see an smtp-failure log to check.  If there are any features  
> for
> debugging I can turn on in MM please let me know.  Lock logging is the
> only thing I could find initially.
Although you do not say so, I have the impression that you have spotted  
nothing unusual in MM's $prefix/logs/error log and SMTP outbound  
traffic is not failing as nothing is making it into the  
$prefix/logs/smtp-failure log. Again you did not confimr this but I  
assume that the local MTA is acting as the outgoing SMTP server for  
Mailman?

If, when the qrunner is started by cron each minute, the new instance  
finds that an earlier incarnation is still running, the new instance  
logs it could not get the lock and exits. So logging about being unable  
to get the qrunner lock is not necessarily that unusual.

Do the log entries in the smtp log give any indication of it taking a  
long time for MM to pass each outgoing messages to the MTA via  
SMTPDirect. The log entries should show the time taken to get each post  
out.

You can get extra detail logged to the MM $prefix/logs/smtp log about  
the progress of delivery by MM to the MTA via SMTPDirect by removing  
some comments on lines in the file  
$prefix/Mailman/Handlers/SMTPDirect.py. There is a function near the  
end called 'deliver' (this is the last function def'ed in the file) and  
in it there are four lines commented out with ## as the first two  
characters. Take a copy of this file as backup and delete the leading  
hash characters. This will not take with any running instance of the  
qrunner which you may have to kill.

This smtp log should now show more sub-transactions of the overall SMTP  
handoff for any given piece of outgoing mail.

> On Sun, 24 Aug 2003, Richard Barrett wrote:
>
>>
>> On Sunday, August 24, 2003, at 12:51  am, Andrew Falgout wrote:
>>
>>> I'm currently running mailman-2.0.13-1 eith sendmail-8.11.6-25.72.
>>> Posts
>>> are processed very fast, but delivery is more than slow.  It takes  
>>> 1-2
>>> minutes per post to delivery mail.  Evaluating what is going on in  
>>> the
>>> queue is a little more than cumbersome.  Over the course of 3-4 days
>>> mails
>>> have been piling up in the qfiles directory to the tune of 4000 mails
>>> and
>>> waiting.  There are mails 3 days old waiting to go out.  I'm looking
>>> for
>>> ways to evaluate what is in the queue, why it's in the queue, and how
>>> to
>>> tell qrunner to report more information on what it's thinking.  Like
>>> perhaps the msgid of the email it's trying to process at this time.
>>>
>> Are you using the local sendmail instance as the outgoing SMTP server?
>>
>> Regardless, what do the Mailman post, smtp and smtp-failure logs show
>> about mail being handled by the system. How does this tie in with the
>> MTA's mail log.
>>
>> If your outgoing mail is being handled by the local sendmail MTA then  
>> I
>> would normally expect Mailman to handoff to the MTA without much  
>> delay.
>> If MM is having problems handing off to sendmail then you might find
>> some evidence of this in the smtp-failure log.
>>> --   
>>> Thank You,
>>> Andrew
>>>
>>> -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+- 
>>> +-
>>> +-+-+-
>>> -      Andrew Wilson Falgout (RHCE)               Linux, E Pluribus
>>> UNIX!  -
>>> -      Systems Admin  II, TIU               Office:  (601) 266-5603
>>>     -
>>> -      University of Southern Mississippi
>>>     -
>>> -                                              Fax:  (601) 266-6788
>>>     -
>>> -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
>>> +-+-+-+
>>
>
> -- 
> Thank You,
> Andrew
>
> -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+-+- 
> +-+-+-
> -      Andrew Wilson Falgout (RHCE)               Linux, E Pluribus  
> UNIX!  -
> -      Systems Admin  II, TIU               Office:  (601) 266-5603     
>     -
> -      University of Southern Mississippi                               
>     -
> -                                              Fax:  (601) 266-6788     
>     -
> -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- 
> +-+-+-+
-----------------------------------------------------------------------
Richard Barrett                               http://www.openinfo.co.uk