[Mailman-Developers] Fwd: Fwd: How To change queue process method

[Maickel Pandie]

My MTA is postfix

Mark Sapiro wrote: "You could also just stop Mailman, deliver a lot of
mail and then start
Mailman."
Thanks I will try that

Question again:
1. am I right that all of new email who comes to the server are queued
in incoming queue by incomingrunner and the step is like this?
"new email"----->Incoming queue----->Outgoing queue----> MTA queue
                   ^                            ^                            ^
          Incomingrunner        Outgoingrunner               ?

2. in what queue to most posibility a long email queue will happen if
I do what Mark Sapiro suggested?

3. can I maintain what email is thrown first between Incoming queue to
Outgoing queue?

4. what runner should I edit if I want to make number 3 happen?

5. am I right that switchboard.py is always use when there are email
moving from one queue to another queue to maintain FIFO in all of
queue in mailman so if I want to change the FIFO I just have to edit
switcboard.py?

Please tell me if I wrong and give your suggestion? Thanks

---------- Forwarded message ----------
From: Ian Eiloart <iane at sussex.ac.uk>
Date: 2008/6/9
Subject: Re: [Mailman-Developers] Fwd:  How To change queue process method
To: Maickel Pandie <maickel.pandie at gmail.com>, mailman-developers at python.org




--On 6 June 2008 15:23:57 +0700 Maickel Pandie <maickel.pandie at gmail.com> wrote:

> Thanks Mark and Ian for your suggestion,
> I want ask again,
> 1. Is queue runners like Incomingqueue runner so I have to make more
> Incomingqueue runner?

yes
>
> 2. How can I made more queue runners?

For example, put this in Mailman/mm_cfg.py :

QRUNNERS = [
  ('ArchRunner',     4), # messages for the archiver
  ('BounceRunner',   4), # for processing the qfile/bounces directory
  ('CommandRunner',  4), # commands and bounces from the outside world
  ('IncomingRunner', 4), # posts from the outside world
  ('NewsRunner',     4), # outgoing messages to the nntpd
  ('OutgoingRunner', 4), # outgoing messages to the smtpd
  ('VirginRunner',   4), # internally crafted (virgin birth) messages
  ('RetryRunner',    4), # retry temporarily failed deliveries
  ]

the number must be a power of 2, so if you want to go higher, try 8 or
16. Perhaps it's only required for OutgoingRunner and RetryRunner.

The performance improvement that I saw after doing this was so huge
that I can't believe it isn't standard. Essentially it means that one
big delivery job doesn't block the rest of the lists.

> 3. Can I make more queue pipe so incoming email is deliver to pipe
> depend on their size and domain destination?

I don't know. It would depend on your MTA, and you haven't said what
that is. If it isn't Exim, I can't help.

> 4 If I bombard my mailing list server using email stress test can I
> make a lot of waiting process in queue?

Probably not without queueing lots of mail to third party domains.
You'd want to be very careful not to spam people.

> Thanks
>
> ---------- Forwarded message ----------
> From: Ian Eiloart <iane at sussex.ac.uk>
> Date: 2008/6/4
> Subject: Re: [Mailman-Developers] How To change queue process method
> To: Mark Sapiro <mark at msapiro.net>, Maickel Pandie
> <maickel.pandie at gmail.com>, mailman-developers at python.org
>
>
>
>
> --On 3 June 2008 11:51:31 -0700 Mark Sapiro <mark at msapiro.net> wrote:
>
>> Maickel Pandie wrote:
>>>
>>> I had research about how fast MTA (postfix) can deliver a lot of email
>>> from mailman.
>>> The one that I want to change is queue process method from FIFO to
>>> size based(sum of domain in the list x size of email),
>>> I have a list that contain the sum of domain each mailing list.
>>> I want Mailman prefer sending email that have less domain destination
>>> and small size of email.
>>>
>>> Can you tell me what file I have to edit so mailman can decide what
>>> email is thrown to postfix first?
>>
>>
>> You would need to modify the Switchboard.files() method in module
>> Mailman/Queue/Switchboard.py.
>>
>> Think hard before you move away from FIFO queue processing. If your out
>> queue is not backlogged, it will essentially be processed FIFO in any
>> case because each time you look there will only be one or two entries
>> to process. If it is backlogged and you process in any but FIFO order,
>> you run the risk of leaving non-preferred entries unprocessed for very
>> long times.
>>
>> Also, your specific strategy would penalize larger (perhaps more
>> popular) lists by delaying posts to those lists in favor of those to
>> smaller (perhaps obscure) lists. Is this really what you want to do?
>
> A better solution might be to configure Mailman to use more queue
> runners. We saw a massive performance increase when we did that,
> because a single large delivery would not hold up other quicker
> deliveries.
>
> --
> Ian Eiloart
> IT Services, University of Sussex
> x3148
> _______________________________________________
> Mailman-Developers mailing list
> Mailman-Developers at python.org
> http://mail.python.org/mailman/listinfo/mailman-developers
> Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
> Searchable Archives:
> http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe:
> http://mail.python.org/mailman/options/mailman-developers/iane%40sussex.a
> c.uk
>
> Security Policy:
> http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp



--
Ian Eiloart
IT Services, University of Sussex
x3148