Creation/deletion of lists through-the-web
Last night I got inspired to add back the through-the-web creation and deletion of mailing lists. I've now got this working for Postfix and can work with other MTAs with a little help from y'all. I will soon be checking all this code into the 2.1 codebase, so watch mailman-checkins shortly. (And hey, it only took 24 hours from inception to completion! :).
Apologies in advance for the length of this message.
First, there is a new pseudo-role[1] called `list creator'. There is a separate, global list creator password, similar to but different than the site admin password. This is because list creation is a site-wide operation, but I don't want to have to give out the site admin password to someone who may only have list creation duties. Needless to say, the site admin can create lists through the web too. You set the list creator's password by using mmsitepass with the -c option.
Let's say Mailman is installed at http://yoursite.com/mailman, then the list creation page is at http://yoursite.com/mailman/create which is linked to from the admin overview page (but not the listinfo overview page).
The create page requires you to enter the name of your list, the email address of the initial list owner, the initial list password (with confirmation), and the list creator's password. There is then a "Create" button to submit the form.
Assuming everything checks out, the new list is created, and the results page gives you three links to follow from there (go to the list's info page, the list's admin page, or create another list).
Deleting a list is done from the list's admin page. There is now another link under "Other Administrative Activities" that says "Delete this list (requires confirmation)". Click on that and you're brought to a page for confirmation of the list deletion operation. You have a radio button to choose whether the archives should be deleted or not (the default is not), and you're prompted for the list admin password[2]. There's a link to cancel the deletion and return to the list's admin page, and a button to actually do the list deletion.
For bonus, the site administrator can inhibit the ability for individual list owners to delete their lists through the web by setting OWNERS_CAN_DELETE_THEIR_OWN_LISTS = 0 in mm_cfg.py. Do this, and the link is not shown in the "Other Administrative Activities" list, and the rmlist cgi will error out every time. The current default for OWNERS_CAN_DELETE_THEIR_OWN_LISTS is 0, but that may change to 1 before the final release. The disadvantage is that with this variable set to 0, list deletion must be done via the command line script.
Now, how to integrate this with MTAs? One reason why ttw list creation and deletion hasn't been (re-)added to Mailman until now is that you typically have to do some manual and difficult crud like edit an /etc/aliases file and run `newaliases' (as root!). I've figured a way around this with Postfix, and of course Exim can be configured to automatically recognize new mailing lists, so I figured it was time to do it. I'm hoping that Sendmail, Qmail, and other MTA users amongst yourselves will contribute the code to glue this together for other mailers.
Here's how I solved it for Postfix; this is outlined in a new README.POSTFIX file.
Postfix has a configuration variable called alias_maps' which tells it where to look for local delivery address associations. It has another variable called alias_databases' which tells it which files
to rebuild when invoked as newaliases. For each of these you can
specify the type of map database the file is kept as. One of these
choices is `hash', which is really a BSD dbhash file. Python has a
dbhash module which nicely handles reading and writing dbhash files
(and it should be enabled by default in Python 2.x, if your system has
Berkeley DB installed, as most Linux distros ought to).
It's moderately well-published what Postfix expects as entries in the dbhash (and it's easy to figure out by dumping a newaliases generated .db file) so, when creating a new list, Mailman can add the necessary keys and values to make Postfix happy. Let's say Mailman is installed as /home/mailman and writes its new list alias entries to the dbhash file at /home/mailman/data/aliases.db. By adding "hash:/home/mailman/data/aliases" to your Postfix's alias_maps variable, Postfix will automatically deliver to your new list. Deleting is as simple as removing the keys from the dbhash.
Note that you do /not/ want to add this file to alias_databases since newaliases won't be touching it.
So this works great, but there's a catch! Postfix will invoke the filter programs under the uid of the owner of the aliases.db file, unless that's root, in which case it'll invoke it as the user defined in the default_privs variable (by default `nobody'). And of course the gid has to match what you specify in --with-mail-gid or Mailman's mail wrapper will complain.
The trick is to touch the alias.db file as root, set the group owner to mailman, and make sure the file is user and group writable. Now, when Mailman adds new keys to aliases.db, the user ownership will remain as root, so Postfix invokes it correctly as nobody. But because aliases.db is group-owned by mailman, the cgi processes can write to the file. And no setuid-root script in sight. I've tested this and it works like a charm.
(Of course, all this is described in cookbook form without the gory details in README.POSTFIX. Also, I like this approach much better than the luser_relay hack I posted about a while ago.)
There's one last bit of glue, and here's where you come in (I'm speaking to the one of you who is still reading this. :). There's a new variable in Defaults.py.in called MTA which must point to a module in the Mailman/MTA directory. This implements the MTA-specific operations needed when creating or deleting a list. The API is that this module should provide two functions: create() and remove() both of which take a MailList object. They should do whatever is necessary to inform the MTA that it's alias database has changed. For Postfix it's really not a lot of code[3].
For Exim, I predict "MTA = None" in mm_cfg.py will Do The Right Thing, since nothing special has to be done. I've no idea at this time what Sendmail, Qmail, or any other MTA will require, and I'm hoping those of you who use those mailers will be able to donate a module.
Phew, that's it. I need sleep.
I'm very interested in getting some feedback, especially for those hearty souls who can check out the current codebase, and give it a whirl. I'm no doubt forgetting some important details, but it sure has been fun. :)
Enjoy, -Barry
[1] I call it a pseudo-role because eventually this will be a role that can be given to specific users (once there's a Real User Database).
[2] Because list deletion is not handled by the admin.py cgi script, it isn't protected by the normal admin login screen. I could (and may) add this extra authentication step, but I'll probably still keep the list password requirement on the deletion page as a strong confirmation of the list owner's intention.
[3] There's one caveat: because I don't want to have to include a setuid-root script, the Postfix.py module doesn't call `postfix reload'. This command has to be done as root. The tradeoff is that Postfix will take about a minute to recognize its alias database has changed. To me that's an acceptable limitation.
"BAW" == Barry A Warsaw <barry@digicool.com> writes:
BAW> I will soon be checking all this code into the 2.1 codebase,
BAW> so watch mailman-checkins shortly.I'm too tired to finish the checkins tonight. The committed trunk is probably unstable at the moment, so even though you're as flush with excitement as I <wink>, please wait until tomorrow before doing your cvs update.
G'night, -Barry
On Wed, May 09, 2001 at 01:45:36AM -0400, Barry A. Warsaw wrote:
The create page requires you to enter the name of your list, the email address of the initial list owner, the initial list password (with confirmation), and the list creator's password. There is then a "Create" button to submit the form.
What about a 'send list creation message' ? And a 'auto-generate password' button ? In my eyes, typically, the person creating the list won't be the list owner him/herself, so those things make sense to me. And what about the domain the list should be created on ?
Deleting a list is done from the list's admin page. There is now another link under "Other Administrative Activities" that says "Delete this list (requires confirmation)". Click on that and you're brought to a page for confirmation of the list deletion operation. You have a radio button to choose whether the archives should be deleted or not (the default is not), and you're prompted for the list admin password[2]. There's a link to cancel the deletion and return to the list's admin page, and a button to actually do the list deletion.
Why not have the 'delete a list' option be on the list creation page as well (make it the 'site administration page' or something.) Again, this won't make sense everywhere, but I'd *love* to be able to give our Sales dept. the list creator password to have them create and destroy lists, without giving them the site password.
Here's how I solved it for Postfix; this is outlined in a new README.POSTFIX file.
You'll want to add checks for the permissios of this file to bin/check_perms, just to be sure. If not run as root, it can't fix it automatically, but it can whine incessantly.
There's one last bit of glue, and here's where you come in (I'm speaking to the one of you who is still reading this. :).
Hi, Barry ! =)
There's a new variable in Defaults.py.in called MTA which must point to a module in the Mailman/MTA directory. This implements the MTA-specific operations needed when creating or deleting a list. The API is that this module should provide two functions: create() and remove() both of which take a MailList object. They should do whatever is necessary to inform the MTA that it's alias database has changed. For Postfix it's really not a lot of code[3].
For Exim, I predict "MTA = None" in mm_cfg.py will Do The Right Thing, since nothing special has to be done. I've no idea at this time what Sendmail, Qmail, or any other MTA will require, and I'm hoping those of you who use those mailers will be able to donate a module.
Sendmail can work in the same way as Postfix, I believe. I'm not sure if you can tell it to use more than one aliases.db file, but if it can, the only difference is the permissions with which the commands are executed. As far as I know, Sendmail also uses dbhash (hash or btree, depending on how you make them) so no problem there.
There's another dimension to it, though. At XS4ALL we use a central database (currently MySQL, soon to be PostgresSQL) for all email aliases, and the list creation mechanism should insert the appropriate aliases into that as well as into the local database -- and they have to differ slightly. (local alias is, say, 'list@listXX.xs4all.nl', where XX is a growing number, but the global alias should be either list@xs4all.nl or list@domain.com, pointing to list@listXX.xs4all.nl). I could do this fairly easily by adapting the Sendmail (or Postfix) MTA module and placing it in the MTA directory under a different name.
[3] There's one caveat: because I don't want to have to include a setuid-root script, the Postfix.py module doesn't call `postfix reload'. This command has to be done as root. The tradeoff is that Postfix will take about a minute to recognize its alias database has changed. To me that's an acceptable limitation.
A minute ? Heh. I doubt that's ever going to be a problem, most humans take a minute to read the resulting webpage, let alone that they compose and send a message inside that minute.
-- Thomas Wouters <thomas@xs4all.net>
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
- Thomas Wouters
| Sendmail can work in the same way as Postfix, I believe. I'm not sure | if you can tell it to use more than one aliases.db file, but if it | can, the only difference is the permissions with which the commands | are executed. As far as I know, Sendmail also uses dbhash (hash or | btree, depending on how you make them) so no problem there.
IIRC, Sendmail will whine a lot if the group file or any directory up to it is writeable by anybody else than 'trusted' users. But it is certainly possible to have more than one alias file.
--
Tollef Fog Heen Unix _IS_ user friendly... It's just selective about who its friends are.
On Wed, May 09, 2001 at 01:45:36AM -0400, Barry A. Warsaw wrote:
Last night I got inspired to add back the through-the-web creation and deletion of mailing lists. I've now got this working for Postfix and can work with other MTAs with a little help from y'all. I will soon be checking all this code into the 2.1 codebase, so watch mailman-checkins shortly. (And hey, it only took 24 hours from inception to completion! :).
Ain't Python great?
Now, how to integrate this with MTAs? One reason why ttw list creation and deletion hasn't been (re-)added to Mailman until now is that you typically have to do some manual and difficult crud like edit an /etc/aliases file and run `newaliases' (as root!). I've figured a way around this with Postfix, and of course Exim can be configured to automatically recognize new mailing lists, so I figured it was time to do it. I'm hoping that Sendmail, Qmail, and other MTA users amongst yourselves will contribute the code to glue this together for other mailers.
IMHO, the proper solution for sendmail is for the admin to put an :include: in /etc/aliases pointing to /home/mailman/data/aliases, and rebuild that from scratch against the current list of lists every time that list changes.
It's moderately well-published what Postfix expects as entries in the dbhash (and it's easy to figure out by dumping a newaliases generated .db file) so, when creating a new list, Mailman can add the necessary keys and values to make Postfix happy. Let's say Mailman is installed as /home/mailman and writes its new list alias entries to the dbhash file at /home/mailman/data/aliases.db. By adding "hash:/home/mailman/data/aliases" to your Postfix's alias_maps variable, Postfix will automatically deliver to your new list. Deleting is as simple as removing the keys from the dbhash.
Note that you do /not/ want to add this file to alias_databases since newaliases won't be touching it.
You're working at the 'compiled' level there, right?
There's one last bit of glue, and here's where you come in (I'm speaking to the one of you who is still reading this. :). There's a new variable in Defaults.py.in called MTA which must point to a module in the Mailman/MTA directory. This implements the MTA-specific operations needed when creating or deleting a list. The API is that this module should provide two functions: create() and remove() both of which take a MailList object. They should do whatever is necessary to inform the MTA that it's alias database has changed. For Postfix it's really not a lot of code[3].
I suspect you'll need one more: how to get the aliases database rebuilt when it's been changed.
Cheers, -- jra
Jay R. Ashworth jra@baylink.com Member of the Technical Staff Baylink The Suncoast Freenet The Things I Think Tampa Bay, Florida http://baylink.pitas.com +1 727 804 5015
"JRA" == Jay R Ashworth <jra@baylink.com> writes:
JRA> Ain't Python great?(ssshhh! they'll discover our secret weapon. :)
JRA> IMHO, the proper solution for sendmail is for the admin to
JRA> put an :include: in /etc/aliases pointing to
JRA> /home/mailman/data/aliases, and rebuild that from scratch
JRA> against the current list of lists every time that list
JRA> changes.I'm really trying to avoid having to write the sendmail code myself. I just have too many scary flashbacks when I even think about mucking with sendmail. ;}
Now that everything's checked in, folks can see how I'm doing it with Postfix, so I hope eventually someone will contribute a Mailman/MTA/Sendmail.py module. Until then, I'll work it so it does something simple.
>> Deleting is as simple as removing the keys from the dbhash.
>> Note that you do /not/ want to add this file to alias_databases
>> since newaliases won't be touching it.
JRA> You're working at the 'compiled' level there, right?If I understand what you're asking, yes. IOW, I don't muck about in the plain text aliases file, but I write directly to the file that newaliases would produce.
>> They should do whatever is necessary to inform the MTA that
>> it's alias database has changed. For Postfix it's really not a
>> lot of code[3].
JRA> I suspect you'll need one more: how to get the aliases
JRA> database rebuilt when it's been changed.That's the beauty of the Postfix approach: I'm modifying the database directly so it doesn't need to be rebuilt (i.e. no need to run newaliases). I don't even have to notify Postfix since it'll notice the change after one minute.
"TW" == Thomas Wouters <thomas@xs4all.net> writes:
TW> What about a 'send list creation message' ? And a
TW> 'auto-generate password' button ? In my eyes, typically, the
TW> person creating the list won't be the list owner him/herself,
TW> so those things make sense to me.The list creation message is always sent currently. Does it make sense to be able to inhibit that?
The auto-generate password button is a good idea, I'll add that.
TW> And what about the domain the list should be created on ?The domain in the url to the `create' script! Note that bin/newlist does need a domain switch, since the context can't be determined when run from the command line, but the create cgi doesn't have that problem. E.g. if I want to create a new list on python.org, I'd visit http://mail.python.org/mailman/create but if I wanted to create a new list on zope.org (a virtual host of a shared Mailman installation), I'd visit http://mail.zope.org/mailman/create.
TW> Why not have the 'delete a list' option be on the list
TW> creation page as well (make it the 'site administration page'
TW> or something.)I thought about that, but it makes things too complicated. E.g. now that global page has to have some pull down menu or other way to specify which list you want to delete. List deletion is list-centric, so it's okay to have it hang off the list admin page (really, it's that the rmlist cgi script requires a list name in the url).
TW> Again, this won't make sense everywhere, but I'd *love* to be
TW> able to give our Sales dept. the list creator password to have
TW> them create and destroy lists, without giving them the site
TW> password.That's a great idea, and I'll make the change so that list deletion follows this password chain: list-owner, global list-creator, site admin.
>> Here's how I solved it for Postfix; this is outlined in a new
>> README.POSTFIX file.
TW> You'll want to add checks for the permissios of this file to
TW> bin/check_perms, just to be sure. If not run as root, it can't
TW> fix it automatically, but it can whine incessantly.Good point! This should probably be done in the Mailman/MTA/Foo.py module so MTA-specific adaptors can do whatever additional checks are necessary.
>> There's one last bit of glue, and here's where you come in (I'm
>> speaking to the one of you who is still reading this. :).
TW> Hi, Barry ! =)Hi Thomas! :)
TW> There's another dimension to it, though. At XS4ALL we use a
TW> central database (currently MySQL, soon to be PostgresSQL) for
TW> all email aliases, and the list creation mechanism should
TW> insert the appropriate aliases into that as well as into the
TW> local database -- and they have to differ slightly. (local
TW> alias is, say, 'list@listXX.xs4all.nl', where XX is a growing
TW> number, but the global alias should be either list@xs4all.nl
TW> or list@domain.com, pointing to list@listXX.xs4all.nl). I
TW> could do this fairly easily by adapting the Sendmail (or
TW> Postfix) MTA module and placing it in the MTA directory under
TW> a different name.I think that's the right approach.
>> [3] There's one caveat: because I don't want to have to include
>> a setuid-root script, the Postfix.py module doesn't call
>> `postfix reload'. This command has to be done as root. The
>> tradeoff is that Postfix will take about a minute to recognize
>> its alias database has changed. To me that's an acceptable
>> limitation.
TW> A minute ? Heh. I doubt that's ever going to be a problem,
TW> most humans take a minute to read the resulting webpage, let
TW> alone that they compose and send a message inside that minute.My thinking exactly! -Barry
I hesitate to bring this up since Barry has currently produced a front end for existing mailman functionality rather than rehashing things... but one of the things I keep being asked about is better domain support....
I think that in general people wish to be able to have multiple domains on a machine and have *distinct* lists of the form list-name@domain1 list-name@domain2
Personally I am actually pretty happy with the current behaviour, and it could be very painful if the case where a current list sometimes gets requests on more than one domain was broken to handle this new case...
Nigel.-- [ Nigel Metheringham Nigel.Metheringham@InTechnology.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] [ - Comments in this message are my own and not ITO opinion/policy - ]
"NM" == Nigel Metheringham <Nigel.Metheringham@InTechnology.co.uk> writes:
NM> Personally I am actually pretty happy with the current
NM> behaviour, and it could be very painful if the case where a
NM> current list sometimes gets requests on more than one domain
NM> was broken to handle this new case...I think it would have to be broken. The `quick-fix' approach I've been mulling would involve encoding the domain into the path to the list's config.db file. I think it would be pretty painful to do that and arrange for lists to span multiple domains.
But then again, maybe not. If the list's directory lived in $mailman/lists then it would span, but if it lived in $mailman/lists/$domain it would not span.
Note, I'm no where near complete in my thinking on this, except to say that I believe the current situation is liveable and I've got other things higher up on my TODO list.
-Barry
On Wed, May 09, 2001 at 12:27:49PM -0400, Barry A. Warsaw wrote:
JRA> IMHO, the proper solution for sendmail is for the admin to JRA> put an :include: in /etc/aliases pointing to JRA> /home/mailman/data/aliases, and rebuild that from scratch JRA> against the current list of lists every time that list JRA> changes.I'm really trying to avoid having to write the sendmail code myself. I just have too many scary flashbacks when I even think about mucking with sendmail. ;}
It seems pretty plain to me. Just expand the "list of lists" though a text filter that explodes the list names into what the doco says the alias entry needs to look like.
Now that everything's checked in, folks can see how I'm doing it with Postfix, so I hope eventually someone will contribute a Mailman/MTA/Sendmail.py module. Until then, I'll work it so it does something simple.
See above. :-) I can't code, but I *can* design...
JRA> You're working at the 'compiled' level there, right?If I understand what you're asking, yes. IOW, I don't muck about in the plain text aliases file, but I write directly to the file that newaliases would produce.
Thought so. Forgive me, my revered senior hat, but that's a bad design. You've put yourself at their mercy by going behind their API...
JRA> I suspect you'll need one more: how to get the aliases JRA> database rebuilt when it's been changed.That's the beauty of the Postfix approach: I'm modifying the database directly so it doesn't need to be rebuilt (i.e. no need to run newaliases). I don't even have to notify Postfix since it'll notice the change after one minute.
See above. ;-)
TW> And what about the domain the list should be created on ?The domain in the url to the `create' script! Note that bin/newlist does need a domain switch, since the context can't be determined when run from the command line, but the create cgi doesn't have that problem. E.g. if I want to create a new list on python.org, I'd visit http://mail.python.org/mailman/create but if I wanted to create a new list on zope.org (a virtual host of a shared Mailman installation), I'd visit http://mail.zope.org/mailman/create.
Oh... mailman *does* get that right? Cool; missed that.
If it knows that, though, how come my mails go out with the wrong From line? Doesn't it set the from line to the proper virtual domain? Or is my configuration wrong?
Cheers, -- jra
Jay R. Ashworth jra@baylink.com Member of the Technical Staff Baylink The Suncoast Freenet The Things I Think Tampa Bay, Florida http://baylink.pitas.com +1 727 804 5015
"JRA" == Jay R Ashworth <jra@baylink.com> writes:
>> I'm really trying to avoid having to write the sendmail code
>> myself. I just have too many scary flashbacks when I even
>> think about mucking with sendmail. ;}
JRA> It seems pretty plain to me. Just expand the "list of lists"
JRA> though a text filter that explodes the list names into what
JRA> the doco says the alias entry needs to look like.That's not the difficult part. What I'm purposefully avoiding is having to install sendmail, and /test/ the interoperability. There are all kinds of dark corners lurking there that only someone who uses and knows sendmail well enough will be able to solve. As I've done for Postfix.
>> JRA> You're working at the 'compiled' level there, right?
>> If I understand what you're asking, yes. IOW, I don't muck
>> about in the plain text aliases file, but I write directly to
>> the file that newaliases would produce.
JRA> Thought so. Forgive me, my revered senior hat, but that's a
JRA> bad design. You've put yourself at their mercy by going
JRA> behind their API...I disagree. Postfix documents all this, so I consider it part of their API. And the fact that they have alias_maps /and/ alias_databases shows that they intend for external applications to create and manage maps that Postfix will consult.
>> cgi doesn't have that problem. E.g. if I want to create a new
>> list on python.org, I'd visit
>> http://mail.python.org/mailman/create but if I wanted to create
>> a new list on zope.org (a virtual host of a shared Mailman
>> installation), I'd visit http://mail.zope.org/mailman/create.
JRA> Oh... mailman *does* get that right? Cool; missed that.
JRA> If it knows that, though, how come my mails go out with the
JRA> wrong From line? Doesn't it set the from line to the proper
JRA> virtual domain? Or is my configuration wrong?That's almost invariably a sending-MTA configuration issue. I know for a fact that Sendmail used to munge From: headers according to its own whims. You could set all the headers properly to your hearts content, and if Sendmail was misconfigured, they'd be broken when the end user gets them.
I'll note that python.org and zope.org are a shared Mailman, virtual host arrangement, and it's all worked perfectly with both Postfix, and now Exim.
-Barry
On Wed, May 09, 2001 at 01:40:19PM -0400, Barry A. Warsaw wrote:
JRA> It seems pretty plain to me. Just expand the "list of lists" JRA> though a text filter that explodes the list names into what JRA> the doco says the alias entry needs to look like.That's not the difficult part. What I'm purposefully avoiding is having to install sendmail, and /test/ the interoperability. There are all kinds of dark corners lurking there that only someone who uses and knows sendmail well enough will be able to solve. As I've done for Postfix.
Ah.
JRA> Thought so. Forgive me, my revered senior hat, but that's a JRA> bad design. You've put yourself at their mercy by going JRA> behind their API...I disagree. Postfix documents all this, so I consider it part of their API. And the fact that they have alias_maps /and/ alias_databases shows that they intend for external applications to create and manage maps that Postfix will consult.
Oh. Didn't realize that. As JC notes, though, there's a problem in reverse.
>> cgi doesn't have that problem. E.g. if I want to create a new >> list on python.org, I'd visit >> http://mail.python.org/mailman/create but if I wanted to create >> a new list on zope.org (a virtual host of a shared Mailman >> installation), I'd visit http://mail.zope.org/mailman/create. JRA> Oh... mailman *does* get that right? Cool; missed that. JRA> If it knows that, though, how come my mails go out with the JRA> wrong From line? Doesn't it set the from line to the proper JRA> virtual domain? Or is my configuration wrong?That's almost invariably a sending-MTA configuration issue. I know for a fact that Sendmail used to munge From: headers according to its own whims. You could set all the headers properly to your hearts content, and if Sendmail was misconfigured, they'd be broken when the end user gets them.
I'll note that python.org and zope.org are a shared Mailman, virtual host arrangement, and it's all worked perfectly with both Postfix, and now Exim.
Noted. Which do you like better?
Cheers, -- jra
Jay R. Ashworth jra@baylink.com Member of the Technical Staff Baylink The Suncoast Freenet The Things I Think Tampa Bay, Florida http://baylink.pitas.com +1 727 804 5015
"JRA" == Jay R Ashworth <jra@baylink.com> writes:
>> I disagree. Postfix documents all this, so I consider it part
>> of their API. And the fact that they have alias_maps /and/
>> alias_databases shows that they intend for external
>> applications to create and manage maps that Postfix will
>> consult.
JRA> Oh. Didn't realize that. As JC notes, though, there's a
JRA> problem in reverse.Not really, because I submit that you shouldn't have to touch the plain text Mailman-specific aliases file. You shouldn't be mixing non-Mailman aliases into this file (and don't need to since Postfix will happily consult any number of files via alias_maps), and Mailman can be taught to DTRT for the Mailman-related aliases.
>> I'll note that python.org and zope.org are a shared Mailman,
>> virtual host arrangement, and it's all worked perfectly with
>> both Postfix, and now Exim.
JRA> Noted. Which do you like better?I'm mostly agnostic here. I chose Postfix years ago for the old CNRI machines when I was just frustrated beyond my breaking point with Sendmail (you don't even want to hear about our even older, internal, and hand-crufted mmdf installation). Once I learned it I had little reason - or really, time - to change.
When we (and our lists ;) moved to Digital Creations, and we integrated the python.org and zope.org domains, I was able to pawn^H^H^H^H, er, hand off the specific MTA duties to the zope.org webmaster. He was very comfortable with Exim and since I was load-shedding, I wasn't in a position to argue. Exim's done the job quite admirably since then. And it's very nice that Exim is so easily integrated with Mailman!
-Barry
On Wed, 9 May 2001 12:27:49 -0400 Barry A Warsaw <barry@digicool.com> wrote:
If I understand what you're asking, yes. IOW, I don't muck about in the plain text aliases file, but I write directly to the file that newaliases would produce.
Then what happens when I need to manually touch that alias file for some reason (eg I edit the aliases to insert demime in the pipeline) and have postfix rebuild it? You need to touch both the plain text alias file and the DBM.
-- J C Lawrence claw@kanga.nu ---------(*) http://www.kanga.nu/~claw/ The pressure to survive and rhetoric may make strange bedfellows
"JCL" == J C Lawrence <claw@kanga.nu> writes:
>> If I understand what you're asking, yes. IOW, I don't muck
>> about in the plain text aliases file, but I write directly to
>> the file that newaliases would produce.
JCL> Then what happens when I need to manually touch that alias
JCL> file for some reason (eg I edit the aliases to insert demime
JCL> in the pipeline) and have postfix rebuild it? You need to
JCL> touch both the plain text alias file and the DBM.Two suggestions:
If you want to insert demime in front of the pipeline for all lists, then edit Mailman/MTA/Utils.py to generate whatever expansion of the aliases you want. Currently these expand to
mylist: "|/path/to/mailman/mail/wrapper post mylist" mylist-admin: "|/path/to/mailman/mail/wrapper mailowner mylist" mylist-request: "|/path/to/mailman/mail/wrapper mailcmd mylist" mylist-owner: mylist-admin
If you only want to insert demime in front of some lists and not others, then I would suggest writing a custom module for the Mailman/MTA directory and using that instead (i.e. setting MTA = 'MyCustomMTA' in mm_cfg.py).
That having been said, I'm adding a command line script (bin/genaliases) that will re-generate both the dbhash alias file and the plain-text alias file based on the list of mailing lists.
What I don't want to do is use the plain text file as the canonical database file for integration with the MTA. I fear the grepping and cut-n-paste that would have to be done programmatically would be too fragile. And then you still have to get newaliases run, which poses problems in its own right.
Ideally, none of this would be necessary because the MTA would just have a single hook for it to auto-recognize mailing lists. Exim does it well, and it looks like David Champion's got a good solution for Sendmail. I'll follow up on the alternative Postfix solution shortly, in a separate email.
-Barry
P.S. genaliases and Mailman/MTA/Utils.py will be checked in soon.
On Thu, May 10, 2001 at 01:10:46PM -0400, Barry A. Warsaw wrote:
What I don't want to do is use the plain text file as the canonical database file for integration with the MTA. I fear the grepping and cut-n-paste that would have to be done programmatically would be too fragile. And then you still have to get newaliases run, which poses problems in its own right.
Well, I *had* suggested making the admin do *1* :include: in the alias file that sendmail knows about, to a file tha mailman *owns*...
Cheers, -- jra
Jay R. Ashworth jra@baylink.com Member of the Technical Staff Baylink The Suncoast Freenet The Things I Think Tampa Bay, Florida http://baylink.pitas.com +1 727 804 5015
"JRA" == Jay R Ashworth <jra@baylink.com> writes:
JRA> Well, I *had* suggested making the admin do *1* :include: in
JRA> the alias file that sendmail knows about, to a file tha
JRA> mailman *owns*...Which makes it a little easier to control the contents of the file, but not foolproof. That's not counting getting newaliases to run under the proper permissions (which still has to be done with :include: files right?)
I still like David's solution best for Sendmail.
Cheers, -Barry
At 02:48 PM 5/10/01 -0400, Barry A. Warsaw wrote:
I still like David's solution best for Sendmail.
His solution is slick, but it requires you run mailman lists off of a special hostname. Granted, many of us do, but not everyone. Heck, my mail mailman server has several lists.whoever domains, but also has some orphan lists not claimed by a particular organization that list under the machines own hostname. Which is where mailman insists on creating them by default normally...
On Thu, 10 May 2001 13:10:46 -0400 Barry A Warsaw <barry@digicool.com> wrote:
"JCL" == J C Lawrence <claw@kanga.nu> writes:
If I understand what you're asking, yes. IOW, I don't muck about in the plain text aliases file, but I write directly to the file that newaliases would produce.
JCL> Then what happens when I need to manually touch that alias file JCL> for some reason (eg I edit the aliases to insert demime in the JCL> pipeline) and have postfix rebuild it? You need to touch both JCL> the plain text alias file and the DBM.
Two suggestions:
- If you want to insert demime in front of the pipeline for all lists, then edit Mailman/MTA/Utils.py to generate whatever expansion of the aliases you want. Currently these expand to
...
- If you only want to insert demime in front of some lists and not others, then I would suggest writing a custom module for the Mailman/MTA directory and using that instead (i.e. setting MTA = 'MyCustomMTA' in mm_cfg.py).
That's taking something which is simple and obvious, which every SysAdm out there who has ever touched mail on a *nix system understands editing alias files, and then turning it into something special cased for Mailman.
Bad.
That having been said, I'm adding a command line script (bin/genaliases) that will re-generate both the dbhash alias file and the plain-text alias file based on the list of mailing lists.
Yeesh. No.
If you touch the DBM file you need to touch the aliases file, always. Otherwise you're in the position of actively working to deceive admins in regard to the state of their mail system (what they can see says one thing, what they can't see says another, and they have to have specialised knowledge to know that there's likely to be a difference).
What I don't want to do is use the plain text file as the canonical database file for integration with the MTA. I fear the grepping and cut-n-paste that would have to be done programmatically would be too fragile.
Hardly.
Put a comment block at the top of the file stating that this file is maintained by the CGI at URL <whatever>. Include a warning that manual editing *might* break supports for this file format. Have newlist and the CGI always use the same formating. As all you are relly interested in are the ^listname:[ \t]*.* lines, they're easy enough to match and edit.
And then you still have to get newaliases run, which poses problems in its own right.
Nope. You do both: touch the text file AND the DBM.
-- J C Lawrence claw@kanga.nu ---------(*) http://www.kanga.nu/~claw/ The pressure to survive and rhetoric may make strange bedfellows
"JCL" == J C Lawrence <claw@kanga.nu> writes:
JCL> If you touch the DBM file you need to touch the aliases file,
JCL> always. Otherwise you're in the position of actively working
JCL> to deceive admins in regard to the state of their mail system
JCL> (what they can see says one thing, what they can't see says
JCL> another, and they have to have specialised knowledge to know
JCL> that there's likely to be a difference).
>> What I don't want to do is use the plain text file as the
>> canonical database file for integration with the MTA. I fear
>> the grepping and cut-n-paste that would have to be done
>> programmatically would be too fragile.
JCL> Hardly.
JCL> Put a comment block at the top of the file stating that this
JCL> file is maintained by the CGI at URL <whatever>. Include a
JCL> warning that manual editing *might* break supports for this
JCL> file format. Have newlist and the CGI always use the same
JCL> formating. As all you are relly interested in are the
JCL> ^listname:[ \t]*.* lines, they're easy enough to match and
JCL> edit.
>> And then you still have to get newaliases run, which poses
>> problems in its own right.
JCL> Nope. You do both: touch the text file AND the DBM.I think I see what you're getting at. Agreed, with the caveat that if the human operator modifies the text file and regens the db file, any f*ckups are outside Mailman's responsibilities to fix (although I'll do my best to fail gracefully).
-Barry
On Thu, 10 May 2001 15:06:35 -0400 Barry A Warsaw <barry@digicool.com> wrote:
I think I see what you're getting at. Agreed, with the caveat that if the human operator modifies the text file and regens the db file, any f*ckups are outside Mailman's responsibilities to fix (although I'll do my best to fail gracefully).
Precisely. Ultimately its the principle of least surprise. What I don't want to see is: Mailman is so lame and broken! Our admin just touched the alias file and now ___ALL___ of our Mailman lists are gone -- AND HE DIDN"T EVEN CHANGE ANYTHING! Stoopid bloody software. Mailman is just so braindead... Or words to that effect. I think we can handle things a little more gracefully than that, both by adding a commented warning and documentation at the top of the alias file (genned on file create and re-added if file touched by Mailman and comments cound missing (cf IMAP and first message)), and by attempting to be graceful (which also means keeping backups at the filesystem level), about all Mailman-driven alias file operations. -- J C Lawrence claw@kanga.nu ---------(*) http://www.kanga.nu/~claw/ The pressure to survive and rhetoric may make strange bedfellows
At 12:27 PM 5/9/01 -0400, Barry A. Warsaw wrote:
"JRA" == Jay R Ashworth <jra@baylink.com> writes:
JRA> Ain't Python great?
(ssshhh! they'll discover our secret weapon. :)
JRA> IMHO, the proper solution for sendmail is for the admin to JRA> put an :include: in /etc/aliases pointing to JRA> /home/mailman/data/aliases, and rebuild that from scratch JRA> against the current list of lists every time that list JRA> changes.
I'm really trying to avoid having to write the sendmail code myself. I just have too many scary flashbacks when I even think about mucking with sendmail. ;}
That's in the cf file. You can specify more than one alias file. Which is probably the cleanest way to do it; you'll still need something, if only a cronjob that runs newaliases - I shudder at the the thought of manipulating the sendmail alias db file directly, particularly since my python 2.0 will flatly not compile with the version of DB that sendmail is using; I suspect it's too new for the DB routines shipped with python.
Note that eric said a while ago that he's deprecating the "auto rebuild alias file" for security reasons, so at some point that functionality will likely go away.
JRA> I suspect you'll need one more: how to get the aliases JRA> database rebuilt when it's been changed.
That's the beauty of the Postfix approach: I'm modifying the database directly so it doesn't need to be rebuilt (i.e. no need to run newaliases). I don't even have to notify Postfix since it'll notice the change after one minute.
Until the first time something goes wrong... Then you'll need a tool to regenerate all the aliases again from scratch. I really don't like the fact that I won't have a text DB source file around...
"TW" == Thomas Wouters <thomas@xs4all.net> writes:
TW> What about a 'send list creation message' ? And a TW> 'auto-generate password' button ? In my eyes, typically, the TW> person creating the list won't be the list owner him/herself, TW> so those things make sense to me.
The list creation message is always sent currently. Does it make sense to be able to inhibit that?
Yes. I currently manually go in and fix some things that just can't be set from defaults easily before letting the admin know. Ideally I'd like a button for "(re)send list creation message now", because what I do is run newlist, hit ctl-z, go off and make my changes, fg it, and let it send the list message...
"RJ" == Ron Jarrell <jarrell@vt.edu> writes:
RJ> That's in the cf file. You can specify more than one alias
RJ> file. Which is probably the cleanest way to do it; you'll
RJ> still need something, if only a cronjob that runs newaliases -
RJ> I shudder at the the thought of manipulating the sendmail
RJ> alias db file directly, particularly since my python 2.0 will
RJ> flatly not compile with the version of DB that sendmail is
RJ> using; I suspect it's too new for the DB routines shipped with
RJ> python.Ah. Python 2.1 should have a much better story here. It doesn't statically link against the db library, and you can install the latest Sleepycat version of BerkeleyDB, and use Robin Dunn's PyBSDDB3 wrapper (pybsddb.sourceforge.net). Yup, that's more work, and it's not ideal, but it ought to work.
RJ> Note that eric said a while ago that he's deprecating the
RJ> "auto rebuild alias file" for security reasons, so at some
RJ> point that functionality will likely go away.Another nail in the coffin for maintaining the aliases in the plain text file. But it looks like David Champion got the right solution for Sendmail.
RJ> Until the first time something goes wrong... Then you'll need
RJ> a tool to regenerate all the aliases again from scratch. I
RJ> really don't like the fact that I won't have a text DB source
RJ> file around...bin/genaliases
>> The list creation message is always sent currently. Does it
>> make sense to be able to inhibit that?
RJ> Yes. I currently manually go in and fix some things that just
RJ> can't be set from defaults easily before letting the admin
RJ> know. Ideally I'd like a button for "(re)send list creation
RJ> message now", because what I do is run newlist, hit ctl-z, go
RJ> off and make my changes, fg it, and let it send the list
RJ> message...I think the easiest thing to do right now is to restore the prompt before notification is sent for the command line script. It's not immediately clear how I should handle this for the create cgi...
-Barry
On 2001.05.09, in <20010509101321.61492@scfn.thpl.lib.fl.us>, "Jay R. Ashworth" <jra@baylink.com> wrote:
that you typically have to do some manual and difficult crud like edit an /etc/aliases file and run `newaliases' (as root!). I've figured a way around this with Postfix, and of course Exim can be configured to automatically recognize new mailing lists, so I figured it was time to do it. I'm hoping that Sendmail, Qmail, and other MTA users amongst yourselves will contribute the code to glue this together for other mailers.
IMHO, the proper solution for sendmail is for the admin to put an :include: in /etc/aliases pointing to /home/mailman/data/aliases, and rebuild that from scratch against the current list of lists every time that list changes.
I've written a mailer (in the sendmail sense) for Mailman. You can define a mailertable entry as: server.name.mil mailman:server.name.mil
And add the mailer to sendmail.cf: MMailman, P=/etc/mail/mm-handler, F=rDFMhlqSu, U=mailman:other, S=EnvFromL, R=EnvToL/HdrToL, A=mm-handler $h $u
...and it's all automagical. No more aliases, no more recompiling map datbases.
You can find it in the archives, or I can mail you the most recent working version.
-- -D. dgc@uchicago.edu NSIT University of Chicago
On Wed, May 09, 2001 at 12:10:49PM -0500, David Champion wrote:
On 2001.05.09, in <20010509101321.61492@scfn.thpl.lib.fl.us>, "Jay R. Ashworth" <jra@baylink.com> wrote:
that you typically have to do some manual and difficult crud like edit an /etc/aliases file and run `newaliases' (as root!). I've figured a way around this with Postfix, and of course Exim can be configured to automatically recognize new mailing lists, so I figured it was time to do it. I'm hoping that Sendmail, Qmail, and other MTA users amongst yourselves will contribute the code to glue this together for other mailers.
IMHO, the proper solution for sendmail is for the admin to put an :include: in /etc/aliases pointing to /home/mailman/data/aliases, and rebuild that from scratch against the current list of lists every time that list changes.
I've written a mailer (in the sendmail sense) for Mailman. You can define a mailertable entry as: server.name.mil mailman:server.name.mil
And add the mailer to sendmail.cf: MMailman, P=/etc/mail/mm-handler, F=rDFMhlqSu, U=mailman:other, S=EnvFromL, R=EnvToL/HdrToL, A=mm-handler $h $u
...and it's all automagical. No more aliases, no more recompiling map datbases.
You can find it in the archives, or I can mail you the most recent working version.
I think that would be *perfect*. Mailers is exactly one of Sendmail's fortes, and very useful if you know how to use them (we do, for various uhm, 'creative systems' :) I'd very much like to test this sometime (but not soon, unfortunately my other work stuff is swamping me) on our list servers!
-- Thomas Wouters <thomas@xs4all.net>
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
"DC" == David Champion <dgc@uchicago.edu> writes:
DC> On 2001.05.09, in <20010509101321.61492@scfn.thpl.lib.fl.us>,
DC> I've written a mailer (in the sendmail sense) for Mailman.
DC> You can define a mailertable entry as: server.name.mil
DC> mailman:server.name.mil
DC> And add the mailer to sendmail.cf: MMailman,
DC> P=/etc/mail/mm-handler, F=rDFMhlqSu, U=mailman:other,
DC> S=EnvFromL, R=EnvToL/HdrToL, A=mm-handler $h $u
DC> ...and it's all automagical. No more aliases, no more
DC> recompiling map datbases.
DC> You can find it in the archives, or I can mail you the most
DC> recent working version.David,
This is really cool! Please send me the latest version of mm-handler so I can add it to the contrib directory. I've updated README.SENDMAIL with the following text; does it accurately describe the steps one should take to integrate Mailman and Sendmail? If not, please let me know what changes should be made.
Thanks, -Barry
-------------------- snip snip -------------------- INTEGRATING SENDMAIL AND MAILMAN
David Champion has contributed a Sendmail mailer which you can use
so that Sendmail will automatically deliver to Mailman mailing
lists without manual intervention (i.e. updated an aliases file or
running newaliases). He suggests adding the following to your
sendmail.cf file:
MMailman, P=/etc/mail/mm-handler, F=rDFMhlqSu, U=mailman:other,
S=EnvFromL, R=EnvToL/HdrToL,
A=mm-handler $h $u
The mm-handler script can be found in the contrib directory; copy
this script to /etc/mail and in your Mailman/mm_cfg.py file, add
the following:
MTA = None
Now Sendmail should automagically deliver to the standard Mailman
aliases for mailing lists.
@ Barry A. Warsaw (barry@digicool.com) :
Last night I got inspired to add back the through-the-web creation and deletion of mailing lists. I've now got this working for Postfix and
Sorry I'm catching this thread late. I've been hacking another way with mailman and postfix, and thought I should share. The idea is outlined below, and the details are on http://listes.rezo.net/comment.php (in French, but there are mostly codes... in code.)
So in postfix create a new virtual domain (listes.rezo.net in my case) within the /etc/postfix/virtual file ; **this domain will be totally used by mailman** ; then use some regexp virtual addresses so that (.*)-post is sent to mailman-post+$1@localhost (.*)-subscribe to mailman-subscribe+$1@localhost and so on. If an adress mathces none of the -action filters, send it to $1-post (so that listname@listes.rezo.net is sent to list-post then to the post programme)
then in the /etc/aliases file define mailman-post: "|/var/lib/mailman/mail/wrapper post $EXTENSION"
and allow the '+' sign as an extension delimiteer in postfix.
Then (shazzam!) any mail coming to the listes.rezo.net domain is processed by the right mailman scripts. Of course we intercept required addresses before the regexp ones, so that postmaster@, abuse@ and root@listes.rezo.net go to root@localhost.
What gives? When I need to create a list, or to delete one, I don't need root privileges or access to any postfix file. I just do the bin/newlist stuff and the list is ready.
Hope this can be helpful.
Fil> So in postfix create a new virtual domain (listes.rezo.net in
Fil> my case) within the /etc/postfix/virtual file ; **this domain
Fil> will be totally used by mailman**Hi Fil,
If I understand correctly, this means your mailing lists have to be advertised as mylist@virt.mydomain.com, right? I.e. you have to expose the virtual host in the domain part of the address.
What if I want to advertise lists as mylist@mydomain.com? Does your approach accomodate that?
Thanks, -Barry
@ Barry A. Warsaw (barry@digicool.com) :
Fil> So in postfix create a new virtual domain (listes.rezo.net in Fil> my case) within the /etc/postfix/virtual file ; **this domain Fil> will be totally used by mailman**Hi Fil,
If I understand correctly, this means your mailing lists have to be advertised as mylist@virt.mydomain.com, right? I.e. you have to expose the virtual host in the domain part of the address.
What if I want to advertise lists as mylist@mydomain.com? Does your approach accomodate that?
Dear Barry,
In fact I do expose them as listname@rezo.net (and not listname@listes.rezo.net). On the rezo.net server (which i separate from listes.rezo.net for efficiency purposes: I don't want my personal mail mixed with heavy list sends) there is a fallback mechanism implemented with a virtual domain served as regexp. **This is very convenient, but not necessary, as the following will show.**
Let's summarize what I use
on rezo.net you find:
/etc/postfix/main.cf:
virtual_maps = hash:/etc/postfix/virtual
/etc/postfix/virtual: rezo.net virtual fil@rezo.net fil # localhost delivery .... # any other fixed addresses and aliases @rezo.net @listes.rezo.net
on listes.rezo.net:
/etc/postfix/main.cf virtual_maps = hash:/etc/postfix/virtual, regexp:/etc/postfix/virtual-regexp
/etc/postfix/virtual: listes.rezo.net virtual postmaster@listes.rezo.net root # intercept the usual stuff root@listes.rezo.net root abuse@listes.rezo.net root
/etc/postfix/virtual-regexp:
# everything that comes here is served (of refused!) by mailman
/^([a-zA-Z0-9\_\-]+)-(post|admin|request|subscribe|unsubscribe)@listes\.rezo\.net$/ mailman-$2+$1
# everything that comes here is for posting to a list, so go back to list-post
/^([a-zA-Z0-9\_\-]+)@listes\.rezo\net$/ $1-post@listes.rezo.net
# everything that comes here is braindead (addresses like toto%2@listes.rezo.net)
/@listes\.rezo\.net$/ abuse@listes.rezo.netHow to do it more simply (i.e. no supplementary domain)
Just take the listes.rezo.net part, replace listes.rezo.net by your domain.net, and add the "fixed aliases" after the abuse@domain.net line in /etc/postfix/virtual
Hope this helps
-- Fil
Postfix has a configuration variable called
alias_maps' which tells it where to look for local delivery address associations. It has another variable calledalias_databases' which tells it which files to rebuild when invoked as newaliases. For each of these you can specify the type of map database the file is kept as. One of these choices is `hash', which is really a BSD dbhash file. Python has a dbhash module which nicely handles reading and writing dbhash files (and it should be enabled by default in Python 2.x, if your system has Berkeley DB installed, as most Linux distros ought to).
It should be mentioned here that postfix and python need to be linked against the same version of libdb as most versions of libdb have incompatible file formats.
One other small caveat is file locking - some versions of libdb (certainly 1.96 and prior) are not safe during updates. The postfix tool that builds libdb hashes uses it's own file locking (check source for details).
Andrew McNamara (System Architect)
connect.com.au Pty Ltd Lvl 3, 213 Miller St, North Sydney, NSW 2060, Australia Phone: +61 2 9409 2117, Fax: +61 2 9409 2111
participants (10)
-
Andrew McNamara -
barry＠digicool.com -
David Champion -
Fil -
J C Lawrence -
Jay R. Ashworth -
Nigel Metheringham -
Ron Jarrell -
Thomas Wouters -
Tollef Fog Heen