[Mailman-Users] Slow Outgoing Queue

[Mark Sapiro]

Harold Paulson wrote:
>
>Each .pck is a separate email message (to one recipient)?  Or each one 
>is a separate post with many recipients?  Is 80-100 items in queue/out 
>a lot?


Each .pck is a message. it could be a post in which case it identifies
all the recipients in one .pck or it could be something like a
password reminder or a subscription confirmation with only one
recipient.

80-100 seems like a lot, but it may not be. Are these files current?
what's in them (see below).


>I have another MLM that typically hands messages off to this MTA at 
>20-30/second, and I wrote it, so I know it isn't fast.  :)
>
>  Does Mailman send messages out serially?

Yes, unless you slice the queue and have multiple runners which I can
see you don't from:

# ps auxwww | grep OutgoingRunner
mailman   3641  1.2  9.5 107680 99240  ??  S    13Oct05 288:04.50 
/usr/local/bin/python /usr/local/mailman/bin/qrunner - 
--runner=OutgoingRunner:0:1 -s


>> The question is why is OutgoingRunner so slow as to create such a
>> backlog or perhaps what happened to dump all these messages into the
>> queue at once.
>
>Is there any way for me to examine these files?  Or do I need to be a 
>Python programmer for that?


bin/show_qfiles and bin/dumpdb can both be used. Give the command with
the --help option for details.

Also, it might be revealing to look at Mailman's 'smtp' log to see
what, if anything, it accomplished between 10:24:25 and 11:14:33
(11:18:45 - 252 seconds).


>> Also, there could be an issue with the qfiles/out directory being
>> physically large even if it only contains a smaller nubmer of entries
>> now, although that should only affect storing new queue entries, not
>> retrieving them. See the post at
>> <http://mail.python.org/pipermail/mailman-users/2005-July/045667.html>
>> for more info on this and some other tips too.
>
>Aha!  There is one very big file in there.  I shall whack it.


I didn't mean a big qfiles/out/*.pck file. I meant if you do

ls -l qfiles

giving something like

total 152
drwxrwsr-x  2 mailman mailman  4096 Nov  8 16:16 archive
drwxrwsr-x  2 mailman mailman  4096 Nov  8 16:16 bounces
drwxrwsr-x  2 mailman mailman  4096 Nov  8 09:34 commands
drwxrwsr-x  2 mailman mailman  4096 Nov  8 16:16 in
drwxrwsr-x  2 mailman mailman  4096 Sep 22  2004 news
drwxrwsr-x  2 apache  mailman 86016 Nov  8 16:16 out
drwxrwsr-x  2 mailman mailman  4096 Nov 15  2004 retry
drwxrwsr-x  2 mailman mailman  4096 Sep 22  2004 shunt
drwxrwsr-x  2 root    mailman 36864 Nov  8 16:16 virgin

Is the size of the out/ directory way big (here it is 21 times the
minimum which is big, but not WAY big). in this example, the out and
virgin queue directories likely got big as a result of a large mass
subscribe with notices generating a large number of messages at once
or something similar.

These queues except shunt and maybe retry are normally empty in a
quiescent Mailman. When the directories themselves are physically
large on disk, it takes extra searching to add new entries because the
entire physical directory must be searched each time to verify the
entry doesn't already exist.

-- 
Mark Sapiro <msapiro at value.net>       The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan