[Mailman-Developers] Mailman queue design problem?
Barry A. Warsaw
Fri, 22 Jun 2001 01:59:28 -0400
>>>>> "CG" == Carson Gaspar <email@example.com> writes:
CG> So I recommend a quick fix of splitting the queue into 3
CG> parts, and a medium term fix of adopting one of the tiered
CG> directory structures (postfix, new sendmail, etc.).
Mailman 2.1's queue is split into at least 7 subdirectories,
- incoming posts from the MTA
- incoming commands (i.e. bounces, -request, -owner, -admin, -join, -leave)
- outgoing mail to the MTA
- outgoing mail to the archiver
- outgoing to the nntpd
- the "shunt" queue (for uncaught system errors)
- the "virgin" queue (for messages crafted by Mailman)
It's fairly easy to add more queues, although I'm not sure what else
would be useful. Some other features of the 2.1 queue system:
1) FIFO order is guaranteed
2) Files are "randomly" distributed in a 120 bit hash space for
no-lock cooperative multiprocessing.
3) Different qrunners can be assigned different priorities (i.e. you
can run your incoming posts and MTA-bound queues more often then
your archiver, nntpd, or command processing queues).
4) The qrunners are long running processes, monitored by a cron
spawned watchdog. Pros: on a warmed up system, delivery occurs
almost immediately, with start up delays amortized over the life of
the qrunner (Python 2.0's cyclic garbage collector helps keep
memory usage under control). Cons: debugging is, er, more
challenging because you have to kill the master qrunner whenever
you (really, I ;) make a change to the code.
All in all, while I have field tested it yet, I think the 2.1 queue
runner is miles ahead of what's in 2.0.