[Mailman-Developers] (no subject)

[J C Lawrence]

I'm working on a design proposal for Mailman v3 and have arrived at
a few questions:

  1) Is there a GPL distributed queue processing system ala IBM's MQ
     about?  I've not been able to find one.  The specific problem
     I'm attempting to solve in a performant manner is:

       How does a given delivery process on a given system obtain
       access to a message it is qualified to deliver with a minimum
       of lock contention or other overhead while preventing any
       other process from attempting to also access the message it
       has acquired from that point forward, but while also allowing
       another process to recover that message should the original
       process or host fail?

     Note that the queue directory and the delivery process may be
     on different systems (eg NFS) and that there may be multiple
     competing queue delivery processes on any given system as well
     as across multiple systems.
  
     NB Needs to be GPL as Mailman is a GNU project.

  2) How much interest is there in optionally supporting VERP?  I've
     no interest in making Mailman v3 VERP-only, but I'd like to see
     it as an option, and specifically, an option which can be
     turned on and off dynamically on a configurable percentage of
     posts basis (eg N% of the messages exploded from an arbitrary
     list post are VERPed).  I've got the basics worked out
     (assuming the local MTA supports plus+addressing), but some of
     the smaller bits are fiddly.

  3) The current design requires messages to have MessageIDs which
     are unique within the range of MessageIDs currently within the
     new Mailman queue system (I'm using MessageIDs as the tag oc
     choice for moderation by email).  To do this, I'm proposing
     that Mailman do a couple new things:

       1) Insert MessageID headers with created values in messages
       that don't contain any MessageID.

       2) Detect collisions within its rather small/arbitrary
       window, and auto-discard/reject messages subsequent messages
       with a duplicate MessageID.  This would not a rigorous dupe
       check, but would only check for dupes against the messages
       already in the Mailman queue (ie received and not yet sent
       back out).

    Any problems in messing with MessageIDs in this way?

    I'm not keen on MessageID re-writing, however its worth noting
    that the above two rules would only come into effect when the
    mail system has already broken down at some level (MUA emitting
    non-unique or no IDs, mail dupes, etc).

  4) While it seems a subtlesmall point, its bugging me.  Given user
     account support, and messages to a given user bouncing, should
     that user be unsubscribed from only that list, or from all
     lists at that site?  Where this is actually bugging me most is
     for virtual domains and whether or not lists in a virtual
     domains should be transparent or opaque to a bounce on a list
     in a different virtual domain?

     The admin in me says, "Hell yes!"  The commercial reality nut
     in me demurrs (think about list hosting for small companies and
     their PR image given transparent virtual hosting).

For those interested the basic model is built upon arbitrary process
queues and pipes.  Also please remember I'm coming up with a
proposal, not a mandate (ie Barry etc haven't seen or commented on
this yet).

-- 
J C Lawrence                                       claw@kanga.nu
---------(*)                        : http://www.kanga.nu/~claw/
--=| A man is as sane as he is dangerous to his environment |=--