I'm working on a design proposal for Mailman v3 and have arrived at a few questions:
Is there a GPL distributed queue processing system ala IBM's MQ about? I've not been able to find one. The specific problem I'm attempting to solve in a performant manner is:
How does a given delivery process on a given system obtain access to a message it is qualified to deliver with a minimum of lock contention or other overhead while preventing any other process from attempting to also access the message it has acquired from that point forward, but while also allowing another process to recover that message should the original process or host fail?
Note that the queue directory and the delivery process may be on different systems (eg NFS) and that there may be multiple competing queue delivery processes on any given system as well as across multiple systems.
NB Needs to be GPL as Mailman is a GNU project.
How much interest is there in optionally supporting VERP? I've no interest in making Mailman v3 VERP-only, but I'd like to see it as an option, and specifically, an option which can be turned on and off dynamically on a configurable percentage of posts basis (eg N% of the messages exploded from an arbitrary list post are VERPed). I've got the basics worked out (assuming the local MTA supports plus+addressing), but some of the smaller bits are fiddly.
The current design requires messages to have MessageIDs which are unique within the range of MessageIDs currently within the new Mailman queue system (I'm using MessageIDs as the tag oc choice for moderation by email). To do this, I'm proposing that Mailman do a couple new things:
Insert MessageID headers with created values in messages that don't contain any MessageID.
Detect collisions within its rather small/arbitrary window, and auto-discard/reject messages subsequent messages with a duplicate MessageID. This would not a rigorous dupe check, but would only check for dupes against the messages already in the Mailman queue (ie received and not yet sent back out).
Any problems in messing with MessageIDs in this way?
I'm not keen on MessageID re-writing, however its worth noting
that the above two rules would only come into effect when the
mail system has already broken down at some level (MUA emitting
non-unique or no IDs, mail dupes, etc).4) While it seems a subtlesmall point, its bugging me. Given user account support, and messages to a given user bouncing, should that user be unsubscribed from only that list, or from all lists at that site? Where this is actually bugging me most is for virtual domains and whether or not lists in a virtual domains should be transparent or opaque to a bounce on a list in a different virtual domain?
The admin in me says, "Hell yes!" The commercial reality nut
in me demurrs (think about list hosting for small companies and
their PR image given transparent virtual hosting).For those interested the basic model is built upon arbitrary process queues and pipes. Also please remember I'm coming up with a proposal, not a mandate (ie Barry etc haven't seen or commented on this yet).
-- J C Lawrence claw@kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=--
While it seems a subtlesmall point, its bugging me. Given user account support, and messages to a given user bouncing, should that user be unsubscribed from only that list, or from all lists at that site? Where this is actually bugging me most is for virtual domains and whether or not lists in a virtual domains should be transparent or opaque to a bounce on a list in a different virtual domain?
The admin in me says, "Hell yes!" The commercial reality nut in me demurrs (think about list hosting for small companies and their PR image given transparent virtual hosting).
I'd like a per-list option to do just that. Some virtual hosts might like that option and others won't.
Ken Kyler
At 5:17 PM -0800 12/11/00, J C Lawrence wrote:
- Is there a GPL distributed queue processing system ala IBM's MQ about? I've not been able to find one.
http://sourceforge.net/projects/queue/
wehn I evaluated it a while back, it wasn't stable on solaris, but it had the functionality I wanted.
- How much interest is there in optionally supporting VERP?
strong here. I did some testing in the last week on a system that is effectively VERPing stuff (but written in perl), which gave me a good idea how to ramp it up to fast volume and get away from the DNS delays and SMTP stuff. it could easily turn into an optional module.
1) Insert MessageID headers with created values in messages that don't contain any MessageID.
that's no problem, although in theory, the MTA should do it for you. The only way I can think of this (if everyone acts properly) happening is someone somehow delivering a message to Mailman that never touches an MTA. I'm not sure that's possible.
2) Detect collisions within its rather small/arbitrary window, and auto-discard/reject messages subsequent messages with a duplicate MessageID. This would not a rigorous dupe check, but would only check for dupes against the messages already in the Mailman queue (ie received and not yet sent back out).
It's not that expensive to keep a hash of message IDs, where the key is the Message-ID, the value is a timestamp. And, say, once a day, you delete records where the timestamp is older than (configurable) days. If you're gong ot dupe check at all, why not do it for real?
Any problems in messing with MessageIDs in this way?
not that I can think of.
(MUA emitting non-unique or no IDs, mail dupes, etc).
it's not the MUA that's responsible for message-iid's, it's the MTA. And every MTA is responsible for adding one if it finds it's missing in the RFCs. So the only way I can see Mailman ever seeing a message without a Message-ID is a local user who somehow uses a local delivery tool like binmail to deliver a posting wtihuout it seeing the local MTA, or if the local MTA is broken. Or if it's a forgery inserted directly into the system somehow. All of which imply the local system is broken or compromised, so IMHO, Mailman doens't need to really worry about it.l
- While it seems a subtlesmall point, its bugging me. Given user account support, and messages to a given user bouncing, should that user be unsubscribed from only that list, or from all lists at that site?
I unsubscribe from the site. I'm sure at some point, an email sent from A might bounce and still be valid if sent from B, but that case is so rare I wouldn't think of wasting time on it, because the only way I can see taht happen (minus broken systems, of course) is someone who decides to try to unsubscribe by blocking a list, isntead of following the directions. And I don't see we need to write code into mailman to help users not follow the instructions.... (grin)
Where this is actually bugging me most is for virtual domains and whether or not lists in a virtual domains should be transparent or opaque to a bounce on a list in a different virtual domain?
since we've talked about a single data store for subscriber data, I think you do it globally. If they really want opaqueness across virtual domains, run mujltiples copies of Mailman. that'll still be an option, after all.
For those interested the basic model is built upon arbitrary process queues and pipes.
which is a nice system -- it's how I finally did my big muther list server, but instead of gnu queue, I'm using QPS.
-- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com)
We're visiting the relatives. Cover us.
Cool, I've read through this thread and I think I see where you're heading. I like a lot of what you guys have posted, and much of it I agree with. I need to break down the separate threads so that each can be captured in the Wiki for posterity.
"JCL" == J C Lawrence claw@kanga.nu writes:
JCL> 4) While it seems a subtlesmall point, its bugging me.
JCL> Given user account support, and messages to a given user
JCL> bouncing, should that user be unsubscribed from only that
JCL> list, or from all lists at that site? Where this is actually
JCL> bugging me most is for virtual domains and whether or not
JCL> lists in a virtual domains should be transparent or opaque to
JCL> a bounce on a list in a different virtual domain?Here's what I've been thinking about. There should be a conceptual user account, with a primary key that may be internal to the system. Users can associate multiple email addresses with their account and can authenticate with the system using any of these addresses as their login. Any one of those email addresses can be deleted at any time with no restrictions, or the entire account can be wiped.
I contend that people will remember (one of) their email addresses better than they will remember a login name they've had to specially craft for the site. Plus, it's likely they've already crafted some unique login @aol.com or @hotmail.com, so why have to potentially craft and remember yet another one just to interact with Mailman at this site? I'm leary about having shorter login names as abbreviations of the email address because today's unique-to-5-chars login may be tomorrow's collision.
Authentication gets trickier when we're pulling users from external databases. Do we authenticate with the userid/password of that external database? Create our own for the mailing list account? I suspect we may need to allow multiple authentication paths for each user.
Once authenticated, a user can edit their options, one of which is a mapping of email addresses in their account to mailing lists. They want to join mailman-users with "barry at digicool.com" but dc-bass with "barry at wooz.org". They can do this on their options pages.
Each mailing list in fact may have a vector of addresses to try for this user. Perhaps there's a default for all lists unless specifically overridden. Perhaps a user can create personal distribution vectors and then can assign a distro vector to a mailing list.
When an address is disabled due to bouncing, Mailman can fallover to the next address in the distro vector. This way, users can plan ahead if they know they're moving. Plus it gives Mailman a way to notify a user when their primary delivery address begins to hard fail.
To add a new address to your distro vector, a confirmation transaction will have to be approved by both an address on the vector already, and the new address being added to the vector. The actual mechanism of this confirmation will be discussed in a separate thread.
So what about virtual domains?
"CVR" == Chuq Von Rospach chuqui@plaidworks.com writes:
CVR> I unsubscribe from the site. I'm sure at some point, an email
CVR> sent from A might bounce and still be valid if sent from B,
CVR> but that case is so rare I wouldn't think of wasting time on
CVR> it, because the only way I can see taht happen (minus broken
CVR> systems, of course) is someone who decides to try to
CVR> unsubscribe by blocking a list, isntead of following the
CVR> directions. And I don't see we need to write code into
CVR> mailman to help users not follow the instructions.... (grin)Agreed.
CVR> since we've talked about a single data store for subscriber
CVR> data, I think you do it globally. If they really want
CVR> opaqueness across virtual domains, run mujltiples copies of
CVR> Mailman. that'll still be an option, after all.I completely agree. In fact, as a user I probably want /more/ globalization of my options and distro vectors, not less. That may be at odds with what some sites want, but too bad. They either run multiple copies or hack the code themselves.
What I mean is, that I'm a member of 10 lists on python.org, 3 or 4 on zope.org, a dozen on SourceForge, and handfuls on other sites. We can excuse the non-Mailman sites their shortsightedness, but I would really love to be able to manage my subscriptions to all those lists in a seamless, transparent manner.
Maybe this is done by editing my accounts with an app on my local system that interfaces to all those Mailmen via CORBA. Maybe it's a Java applet that I configure locally with the list of sites, and it screenscrapes and presents a consolidated view to me. Maybe it only works with a cooperative federation of Mailmen sharing ZEO connections. Whatever. I'm not expecting this to be implemented first, or even ever, but it's my vision of how a user /should/ be able to interact with the system.
-Barry
At 6:03 PM -0500 12/14/00, Barry A. Warsaw wrote:
Here's what I've been thinking about. There should be a conceptual user account, with a primary key that may be internal to the system.
absoldefinitely. My current big machine uses the email address as the primary key in the databases. Makes sense at some level -- until you realize the email address changes (a lot).
So in the redesign I'm doing, I'm assigning a user_id to an account, and it's unique to that account. you can then attach an email address to the account, and not have to worry about it wandering out into the rest of the database so it's easy to update. (one of these minutes I'll finish the key parts of the schema and post it). Assigning multiple addresses to an account, and defining one as the "receiver" address and the rest as poster addresses is a small enhancement from there.
It complicates bounce processing some, but a well-designed search system onto the email address (more on that later, if there's interest) gets around it. turns out you rarely have to do brute force searches for an address if you put a little thought into it.
I contend that people will remember (one of) their email addresses better than they will remember a login name they've had to specially craft for the site.
true. in fact, I probably wouldn't advertise an account/login name. Instead, I'd use a password and any defined email, perhaps with a few carefully chosen heuristics to help find them if they're confused (for instance, users use earthlink.com and earthlink.net interchangeably). You still have problems wehre companies re-arrange their e-mail name space and don't tell the workers (and that happens more often than you might think) but leave in aliases for the old names, but you won't get 100%.
To get into the account, you need one email address attached to it, and the password. To get the passowrd, you need to know any attached email address, and it's sent to that address. That way, they don't need to remember anything, but if they want to, they can.
Authentication gets trickier when we're pulling users from external databases. Do we authenticate with the userid/password of that external database? Create our own for the mailing list account? I suspect we may need to allow multiple authentication paths for each user.
And another option is "none" -- where Mailman is simply the delivery agent for an address system controlled elsewhere and whic users aren't allowed to update via Mailman. Once you start going to external databases, either they're likely to be holding stores for a standard mailman database, or they're likely to be severely restricted access, or read-only from the Mailman point of view.
Each mailing list in fact may have a vector of addresses to try for this user. Perhaps there's a default for all lists unless specifically overridden. Perhaps a user can create personal distribution vectors and then can assign a distro vector to a mailing list.
As a side note -- if we do this, we need to make sure we can assign different addresses to different lists, all under the same account. So if someone wants to put eachlist to a different address, they can... that starts turning into an N x N mapping, so it can get complex (and it implies that the account has an account ID, whic points to 1 -> N email addresses, which each have an email ID, whic is what's used to do the actual subscription. So the schema starts getting complex...)
To add a new address to your distro vector, a confirmation transaction will have to be approved by both an address on the vector already, and the new address being added to the vector.
After the first one, why? (note for future mumuring: leave an interface for the ability to build different validation setups, or allow them to validate via one of many. don't hardware mailbacks as THE validation setup...) -- you have an audit trail back to the person, so if they decide to try to spam someone you know who they are, and who to shoot. As long as you don't lose the authenticity trail, once is all you need (that would, I think, require authenticating another address before allowing deletion of the one that's authenticated, and disabling any account when all of the authenticated addresses are disabled by bounce processing...)
So what about virtual domains?
Um, well... At a high level, it's simply another table in the schema, where you attach a host (and associated UI) around the core of mailman. So you could potentially (he says hopefully) have a list live on multiple domains, or even the same domain under multiple UIs through careful abuse of this interface...
(why do the latter? how about allowing me to have a "test" UI on the domain for development purposes, but having two 'mailman systems' on one virtual host under two URLs?)
What I mean is, that I'm a member of 10 lists on python.org, 3 or 4 on zope.org, a dozen on SourceForge, and handfuls on other sites. We can excuse the non-Mailman sites their shortsightedness, but I would really love to be able to manage my subscriptions to all those lists in a seamless, transparent manner.
but -- maybe the sites don't want that done? For marketing purposes, for identification purposes, for whatever purposes?
I don't even WANT to start thinking about sharing user data across physical machines. Virtual hosts are enough joy here.
I"m of two minds here. One mind sees a reason why virtual hosts don't want to share -- but in that case, do we build this into the system, and if so, how? It's one set of data, and if they need to be left alone, aren't they better off running their own unique instance of mailman? (and can we validate the security of their data? I don't think we want to go there)
But my other mind looks at trying to do all this, and shudders. So I guess I'm in the school, at least right now, of saying "we have one mailman engine, N lists, M vhosts. And for every vhost, there are a subset of those N lists published, but if you access the admin page through that vhost, you get that vhost's UI -- but it's a portal into the global mailman data setup. If they have to be kept separate, run multiple instances of mailman with separate data stores.
The only burble I have with that is -- making sure the user knows where a given list exists in the space, so they can access it if they run into it while doing admin on a site where it's not published. Either that, or (I'd probably prefer it this way) you can't get info on subscribing to a list from other than a vhost it's advertised on, but anythign you're already subscribed to, you can manage. Basically, i guess, I'm treating public lists on other vhosts as private lists on this vhost... (I think that works. yet?))
-- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com)
We're visiting the relatives. Cover us.
CVR> So in the redesign I'm doing, I'm assigning a user_id to an
CVR> account, and it's unique to that account. you can then attach
CVR> an email address to the account, and not have to worry about
CVR> it wandering out into the rest of the database so it's easy
CVR> to update. (one of these minutes I'll finish the key parts of
CVR> the schema and post it). Assigning multiple addresses to an
CVR> account, and defining one as the "receiver" address and the
CVR> rest as poster addresses is a small enhancement from there.Very cool.
CVR> It complicates bounce processing some, but a well-designed
CVR> search system onto the email address (more on that later, if
CVR> there's interest) gets around it. turns out you rarely have
CVR> to do brute force searches for an address if you put a little
CVR> thought into it.Good.
CVR> true. in fact, I probably wouldn't advertise an account/login
CVR> name.I agree.
CVR> Instead, I'd use a password and any defined email, perhaps
CVR> with a few carefully chosen heuristics to help find them if
CVR> they're confused (for instance, users use earthlink.com and
CVR> earthlink.net interchangeably).Hadn't thought about that, and that is a good point.
CVR> You still have problems wehre companies re-arrange their
CVR> e-mail name space and don't tell the workers (and that
CVR> happens more often than you might think) but leave in aliases
CVR> for the old names, but you won't get 100%.Right. I don't think 100% coverage is necessary. I really want to handle the situations where someone knows his email is going to change, or one where they purposely want different lists to deliver to different addresses.
Here's another complication: are the delivery options set per-address or per-list? Maybe I want all deliveries to "barry at wooz" to be digests. Maybe I want lists A, B, and C to be regular deliveries.
CVR> To get into the account, you need one email address attached
CVR> to it, and the password. To get the passowrd, you need to
CVR> know any attached email address, and it's sent to that
CVR> address. That way, they don't need to remember anything, but
CVR> if they want to, they can.Agreed.
CVR> And another option is "none" -- where Mailman is simply the
CVR> delivery agent for an address system controlled elsewhere and
CVR> whic users aren't allowed to update via Mailman. Once you
CVR> start going to external databases, either they're likely to
CVR> be holding stores for a standard mailman database, or they're
CVR> likely to be severely restricted access, or read-only from
CVR> the Mailman point of view.Good point.
CVR> As a side note -- if we do this, we need to make sure we can
CVR> assign different addresses to different lists, all under the
CVR> same account.Yup, absolutely, which is why I posed the config option question above.
CVR> So if someone wants to put eachlist to a
CVR> different address, they can... that starts turning into an N
CVR> x N mapping, so it can get complex (and it implies that the
CVR> account has an account ID, whic points to 1 -> N email
CVR> addresses, which each have an email ID, whic is what's used
CVR> to do the actual subscription. So the schema starts getting
CVR> complex...)I see a level of indirection coming to the rescue. MailingLists have Rosters and Rosters have EmailAddresses which in turn link back to the UserAccount. A MailingList might actually want to deliver to multiple Rosters, which is where I think the umbrella list stuff could be improved. I.e. you have a Roster for mailman-developers and a Roster for mailman-users and mailman-announce contains a computed Roster composed of those first two, along with it's own Roster. Now you send a message out to mailman-announce and everybody gets it (although what do you do about Subject: munging, footer addition List-* headers and the like?). I also see creation of Rosters for list owners, site administrators and so on, so you could do things like compute a Roster for all-list-owners@mysite.com if you had urgent information for your list admins.
CVR> After the first one, why? (note for future mumuring: leave an
CVR> interface for the ability to build different validation
CVR> setups, or allow them to validate via one of many. don't
CVR> hardware mailbacks as THE validation setup...)Agreed (and I have some thoughts on the mailback thread which I'll try to get to separately).
CVR> -- you have an
CVR> audit trail back to the person, so if they decide to try to
CVR> spam someone you know who they are, and who to shoot. As long
CVR> as you don't lose the authenticity trail, once is all you
CVR> need (that would, I think, require authenticating another
CVR> address before allowing deletion of the one that's
CVR> authenticated, and disabling any account when all of the
CVR> authenticated addresses are disabled by bounce processing...)I'm concerned about the scenario where I subscribe to a list, then add your address to my account, then disable my address because I'm "going on vacatin". Now suddenly you're getting flooded with postings that make no sense to you. You don't know anything about Mailman, so you're not even sure where to start complaining. And it could get very annoying very quickly. Sure, once the admins are aware of the problem they can trace it back to me and I'll get slapped around, but in the meantime you're really pissed off. Where if your address was confirmed when I tried to add it, you might still be pretty confused, but you shouldn't be annoyed. (Come to think of it, without protections, this is a nice annoyance spam route or DoS).
What I mean is, that I'm a member of 10 lists on python.org, 3 or 4 on zope.org, a dozen on SourceForge, and handfuls on other sites. We can excuse the non-Mailman sites their shortsightedness, but I would really love to be able to manage my subscriptions to all those lists in a seamless, transparent manner.
CVR> but -- maybe the sites don't want that done? For marketing
CVR> purposes, for identification purposes, for whatever purposes?No doubt. It's what I as a user of Mailman want though. :)
CVR> I don't even WANT to start thinking about sharing user data
CVR> across physical machines. Virtual hosts are enough joy here.No, no, no, neither do I. I was just musing. Not to be seriously considered for 3.0 (if at all!).
CVR> So I guess I'm in the school, at least right now, of saying
CVR> "we have one mailman engine, N lists, M vhosts. And for every
CVR> vhost, there are a subset of those N lists published, but if
CVR> you access the admin page through that vhost, you get that
CVR> vhost's UI -- but it's a portal into the global mailman data
CVR> setup. If they have to be kept separate, run multiple
CVR> instances of mailman with separate data stores.I'm firmly in agreement.
CVR> Either that, or (I'd probably prefer it this way) you can't
CVR> get info on subscribing to a list from other than a vhost
CVR> it's advertised on, but anythign you're already subscribed
CVR> to, you can manage. Basically, i guess, I'm treating public
CVR> lists on other vhosts as private lists on this vhost... (I
CVR> think that works. yet?))Yes, that's what I've been thinking too.
-Barry
On Thu, 14 Dec 2000 19:59:41 -0500 Barry A Warsaw barry@digicool.com wrote:
Here's another complication: are the delivery options set per-address or per-list? Maybe I want all deliveries to "barry at wooz" to be digests. Maybe I want lists A, B, and C to be regular deliveries.
We can generalise this problem into insolvability.
Arguably customisation at the list level is the lowest common demoninator (another other config can be built from there), so we should do that. This is especially pleasant as, should somsone want to do something else they can just take replace the membership tools and do their own thing to provide theiur own model.
Repeat after me:
Mailman is not trying solve all problems!
Mailman is trying to provide a toolkit such that users can solve problems in ways they prefer!
We're providing a toolkit, a set of relationships, and a reference implementation of how they *might* be related an installed in practice. From there its up to SysAdms and end users.
CVR> As a side note -- if we do this, we need to make sure we can CVR> assign different addresses to different lists, all under the CVR> same account.
Yup, absolutely, which is why I posed the config option question above.
Note: We need to allow account customisation for accounts we maintain *AND* for accounts that external agencies maintain. We also need to allow lists to be populated both with accounts that can be customised, and accounts that can't (eg corporate addresses and outside subscriptions).
I actually acount at the point that I think Mailman shouldn't have a default membership implementation, but again, just one or more reference implementations.
-- J C Lawrence claw@kanga.nu ---------(*) http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=--
"JCL" == J C Lawrence claw@kanga.nu writes:
JCL> We're providing a toolkit, a set of relationships, and a
JCL> reference implementation of how they *might* be related an
JCL> installed in practice. From there its up to SysAdms and end
JCL> users.Look at slightly differently, Mailman is a framework with interfaces to key components. Those components can be customized or replaced via subclassing or interface conformance. Either way...
JCL> I actually acount at the point that I think Mailman shouldn't
JCL> have a default membership implementation, but again, just one
JCL> or more reference implementations....it adds up to Mailman providing those reference implementations in an easy to install and use distro, but allowing the flexibility for sites to drop-in replace bits and pieces as necessary.
-Barry
At 6:50 PM -0800 12/14/00, J C Lawrence wrote:
We can generalise this problem into insolvability.
great1 let's define 3.0 done, then, and go have a beer (ducking)
Mailman is not trying solve all problems!
yes, it is. We will merely defer some of them to future releases (or generations)
Mailman is trying to provide a toolkit such that users can solve problems in ways they prefer!
but we need to understand all this enough to be able to write them, even fi we don't write them ourselves, lest those toolkits not do what's needed of them.
I actually acount at the point that I think Mailman shouldn't have a default membership implementation, but again, just one or more reference implementations.
We go back to where I stepped in here (stepped into it?) a few days back. We define a subscriber management interface, splitting Mailman into two pieces: subscriber management and MLM actions. Then the subscriber piece is implemented twice to prove the API -- one by porting the existing Mailman interface to it, and one by (I'd hope) my proposed site-wide authentication scheme to it. Assuming it works for those two cases, it likely works for almose every case, and if someone wants to hook it up to an oracle database via LDAP for corporate mailing lists -- they write another module to the API.
-- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com)
We're visiting the relatives. Cover us.
At 7:59 PM -0500 12/14/00, Barry A. Warsaw wrote:
CVR> Instead, I'd use a password and any defined email, perhaps CVR> with a few carefully chosen heuristics to help find them if CVR> they're confused (for instance, users use earthlink.com and CVR> earthlink.net interchangeably).Hadn't thought about that, and that is a good point.
you learn these the hard way. Guess how many ways there are to misspell hotmail.com...
Right. I don't think 100% coverage is necessary.
My motto: get the first 90% right, then work on the next 90% of what's left. Especially with email, 100% solutions don't exist, because of disasters like lotus notes and other non-conformant systems.
Here's another complication: are the delivery options set per-address or per-list? Maybe I want all deliveries to "barry at wooz" to be digests. Maybe I want lists A, B, and C to be regular deliveries.
Neither, really.
For every account, you can subscribe one or more address to a list. For every address subscribed, you have a set of list options. So a person could get both the messages and digest to separate addresses, and have a third address validated for posting but get nothing. Useful if that person is doing offline munging into a private archive (or if you're using this form to gateway into some scripting system as the admin, where the scripting isn't tied into Mailman. you could then have a single account, but attach all of the gateway addresses to it, and configure each one separately. Much neater administratively.)
I see a level of indirection coming to the rescue. MailingLists have Rosters and Rosters have EmailAddresses which in turn link back to the UserAccount.
Let me think about this in pseudo-SQL terms a sec.
A user creates an account. that is given an acct_id, which is unique to the system. He attaches 1..N email addresses to his account. Each email gets an email_ID, which is also unique to the system, so we now have a 1->mapping from account to a set of email addresses. One or more of these addresses have been validated in some way to guarantee ownership.
question: are email addresses unique to the system? to the user? I'd argue they have to be, if for no other reason than if foo@bar.com is attached to two accounts and someone logs on via it, which acct does he get? So email addresses are unique but you can't use the email address as the primary key because it changes. So any time you add one or change one, you have to validate against uniqueness before accepting it.
from the other side, the admin creates a list, which is assigned a list_id. when a user subscribes to that list, the relationship is between a user's email_id and the list_id, and there's a unique set of preferences attached to that relationship. the only way a list can find out who the user is is to refer back through the list_id to get the account_id, which, frankly, I don't think we want to allow anyway, on privacy issues. you only get the information you need to do the job.
A MailingList might actually want to deliver to multiple Rosters, which is where I think the umbrella list stuff could be improved. I.e. you have a Roster for mailman-developers and a Roster for mailman-users and mailman-announce contains a computed Roster composed of those first two, along with it's own Roster. Now you send a message out to mailman-announce and everybody gets it (although what do you do about Subject: munging, footer addition List-* headers and the like?).
and you need to do duplicate supression, too. I think the headers have to come from the list subscribed to, because that's the list the user will try to ujnsubscribe from, and all the documentation in the world won't explain why headers for mailman-announce won't work for you because you're really on mailman-users.
I also see creation of Rosters for list owners, site administrators and so on, so you could do things like compute a Roster for all-list-owners@mysite.com if you had urgent information for your list admins.
and a meta-list for all subscribers for the same.
But then you get into the issue of who has permission to post to what, and you end up with an authentication database (which is not a bad idea, FWIW.) Especially if you want ot get into whether the site admin wants to allow list admins to set reply-to coercion or not...
CVR> -- you have an CVR> audit trail back to the person, so if they decide to try to CVR> spam someone you know who they are, and who to shoot. As long CVR> as you don't lose the authenticity trail, once is all you CVR> need (that would, I think, require authenticating another CVR> address before allowing deletion of the one that's CVR> authenticated, and disabling any account when all of the CVR> authenticated addresses are disabled by bounce processing...)I'm concerned about the scenario where I subscribe to a list, then add your address to my account, then disable my address because I'm "going on vacatin".
I thought about it at the hockey game, and now I disagree with myself. all addresses are validated. Otherwise, it gets gnarly. First, if you validate an address and add others, and the first bounces -- what do you do? you can't subscribe the others until they'r evalidated. That is the wrong time from user expectation to ask for that. At best, you confuse the hell out of someone.
Second, it opens you up to mailforward attacks (create a hotmail account. Sign up for 900 lists. Forward that account to someone you hate. disappear). At least with validations, a user sees it coming, and knowing they'll get warning, it'll only get used by stupid users...
So yea, on further review, forget i suggested that.
-- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com)
We're visiting the relatives. Cover us.
participants (4)
-
barry＠digicool.com -
Chuq Von Rospach -
J C Lawrence -
Ken Kyler