[Mailman-Developers] Fwd: How To change queue process method

[Ian Eiloart]

--On 6 June 2008 15:23:57 +0700 Maickel Pandie <maickel.pandie at gmail.com> 
wrote:

> Thanks Mark and Ian for your suggestion,
> I want ask again,
> 1. Is queue runners like Incomingqueue runner so I have to make more
> Incomingqueue runner?
yes
> 2. How can I made more queue runners?
For example, put this in Mailman/mm_cfg.py :

QRUNNERS = [
    ('ArchRunner',     4), # messages for the archiver
    ('BounceRunner',   4), # for processing the qfile/bounces directory
    ('CommandRunner',  4), # commands and bounces from the outside world
    ('IncomingRunner', 4), # posts from the outside world
    ('NewsRunner',     4), # outgoing messages to the nntpd
    ('OutgoingRunner', 4), # outgoing messages to the smtpd
    ('VirginRunner',   4), # internally crafted (virgin birth) messages
    ('RetryRunner',    4), # retry temporarily failed deliveries
    ]

the number must be a power of 2, so if you want to go higher, try 8 or 16. 
Perhaps it's only required for OutgoingRunner and RetryRunner.

The performance improvement that I saw after doing this was so huge that I 
can't believe it isn't standard. Essentially it means that one big delivery 
job doesn't block the rest of the lists.

> 3. Can I make more queue pipe so incoming email is deliver to pipe
> depend on their size and domain destination?

I don't know. It would depend on your MTA, and you haven't said what that 
is. If it isn't Exim, I can't help.

> 4 If I bombard my mailing list server using email stress test can I
> make a lot of waiting process in queue?

Probably not without queueing lots of mail to third party domains. You'd 
want to be very careful not to spam people.

> Thanks
>
> ---------- Forwarded message ----------
> From: Ian Eiloart <iane at sussex.ac.uk>
> Date: 2008/6/4
> Subject: Re: [Mailman-Developers] How To change queue process method
> To: Mark Sapiro <mark at msapiro.net>, Maickel Pandie
> <maickel.pandie at gmail.com>, mailman-developers at python.org
>
>
>
>
> --On 3 June 2008 11:51:31 -0700 Mark Sapiro <mark at msapiro.net> wrote:
>
>> Maickel Pandie wrote:
>>>
>>> I had research about how fast MTA (postfix) can deliver a lot of email
>>> from mailman.
>>> The one that I want to change is queue process method from FIFO to
>>> size based(sum of domain in the list x size of email),
>>> I have a list that contain the sum of domain each mailing list.
>>> I want Mailman prefer sending email that have less domain destination
>>> and small size of email.
>>>
>>> Can you tell me what file I have to edit so mailman can decide what
>>> email is thrown to postfix first?
>>
>>
>> You would need to modify the Switchboard.files() method in module
>> Mailman/Queue/Switchboard.py.
>>
>> Think hard before you move away from FIFO queue processing. If your out
>> queue is not backlogged, it will essentially be processed FIFO in any
>> case because each time you look there will only be one or two entries
>> to process. If it is backlogged and you process in any but FIFO order,
>> you run the risk of leaving non-preferred entries unprocessed for very
>> long times.
>>
>> Also, your specific strategy would penalize larger (perhaps more
>> popular) lists by delaying posts to those lists in favor of those to
>> smaller (perhaps obscure) lists. Is this really what you want to do?
>
> A better solution might be to configure Mailman to use more queue
> runners. We saw a massive performance increase when we did that,
> because a single large delivery would not hold up other quicker
> deliveries.
>
> --
> Ian Eiloart
> IT Services, University of Sussex
> x3148
> _______________________________________________
> Mailman-Developers mailing list
> Mailman-Developers at python.org
> http://mail.python.org/mailman/listinfo/mailman-developers
> Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
> Searchable Archives:
> http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe:
> http://mail.python.org/mailman/options/mailman-developers/iane%40sussex.a
> c.uk
>
> Security Policy:
> http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp



-- 
Ian Eiloart
IT Services, University of Sussex
x3148