On Thu, 14 Dec 2000 22:37:51 -0500 Barry A Warsaw <barry@digicool.com> wrote:
"JCL" == J C Lawrence <claw@kanga.nu> writes:
- inbound Message arrives at the MLM 2) authentication Do I > accept it? 3) moderation Does the list accept it?
Remind me again about the difference between 2 and 3, and why 2 is under the purview of the MLM.
- pending Associate a distribution list with message 5) outbound Send it. 5) bounce Demote the subscriber 7) command A combo of #2, #3, and command processing.
I'm not totally sold that 4 needs a process boundary, or that 2, 3, and 4 aren't part of the same process, structured as interfaces in the framework.
Oddly enough I didn't write what I meant to write, and what I meant also wasn't what my own notes and documents say (which happen to be more correct than I was). I'm looking at the following process. First the base model:
Configuration of what exactly happens to a message is done by dropping scrpts/program in specially named directories (it is expected that typically only SymLinks will be dropped (which makes the web interface easy -- just creates and moves symlinks about)).
In general processing of any item will consist of taking that item and iteratively passing it to every script in the appropriately named directory, invoking those scripts in directory sort order (cf SysV init scripts with /etc/rc.# etc). This makes the web config interface easy -- it just varies the numerical prefix on the scripts to enforce a processing order.
Of course each script must follow a defined contract as far arguments, IO, return codes, and other behaviour.
The processing sequence for a message:
Message arrives in inbound queue.
Message is picked up it is passed on to moderation which consists of:
a) extracting a set of meta data from the message and any
associated resources and then associating that meta data with
the message (this is done as an efficiency support ala
pre-processing (very useful for later template expansion))
b) Iteratively passing the message thru all the scripts in the
moderation directory, in order, until either one returns
non-zero or all scripts have been run. All scripts returning
zero means that the message is moved to the pending queue.
Various non-zero returns have other effects ranging from instant
deletion, leaving the message in the inbound queue, moving to
the moderation queue ("holding pen" is more accurate),
auto-bouncing some sort of reply...Some event happens to move a mesasge from the moderation queue to the pending queue (or it goes straight there, bypassing moderation).
Message is found in the pending queue and two things happen in order:
a) It is passed iteratively, with its meta data through every
script in the membership directory. A non-zero return means
that the message stays in pending. The combined output (stdout)
of all membership scripts forms the distribution list for that
message. Upon getting a membership (everything returns zero),
the resultant list is passed thru a specially anemd script (if
it exists) for post-processing (dupe removal, VERP instruction
insertion, domain/MX sorting, etc). The final distribution list
is associated with the message much like the meta data already
is.
b) The message is passed iteratively, with its meta data and
distribution list through the contents of the pre-post
directory. This does whatever it wants to do (archiving,
digests, templating, VERP spit outs, whatever). A non zero
return will leave the message in the pending queue for a later
pass.Finally, having gained a distribution list and been pre-post processed (if there's anything there), the message is moved to the outbound queue with its distribution list.
The mesasge is found in the outbound queue and is handed off to whatever the transport is (MTA, NNTP, whatever).
Now where this gets interesting is that every script and tool along the path above is free to edit the message, edit the meta data, edit the distribution list, to cause the current message to be silently deleted, and/or to crete new or derived messages and to inject them into any other mesasge queue in the system.
So in my view, when Mailman decides that a message can be delivered to a membership list, it's dropped fully formed in an outbound queue.
Yes, with its associated distribution list.
JCL> Not exactly. It drops a mesasge, any relevant meta data, and a JCL> distribution list in the outbound queue. A delivery process JCL> then takes that and does what it will with them (eg VERP, JCL> application of templates, etc).
In Mailman 2.0 the distribution list is also metadata -- it lives in the .db file. Do you see that differently?
I see it as a class of meta data, but specifically one that is bound to and unique to the message in question, not to the list or the list config (consider the previous discussion of of nosy lists). As such a distribution list is genned for every message, and then attached to that message for later editing/delivery.
These processes are not completely independent of Mailman though, e.g. for handling hard errors at smtp transaction time or URL generation for summary digests. Some of these can be handled by re-injection into the message queues (i.e. generate a bounce message and stick it in the bounce queue), but some may need an rpc interface.
JCL> Thus the pending queue above -- it allows a mesasge to undergo JCL> a set of pre-post filters prior to landing in the outbound JCL> queue. Archiving, digests, all sorts of things can happen at JCL> that point.
Maybe I'm taking the term "distribution list" too narrowly in your description about. Do you "list of recipient addresses" or do you mean "internal queue routing destinations", or something else?
I don't understand your question. Given the above, could you rephrase?
-- J C Lawrence claw@kanga.nu ---------(*) http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment=--