Mailman 3 October 2007 - Mailman-Developers

Re: [Mailman-Developers] Improving the archives
by Craig Loomis Nov. 4, 2007

Nov. 4, 2007

Or Re: [Mailman-Developers 10417] Improving the archives I would like to interject and highlight some use cases for stable and predictable IDs. For us, "message IDs" are directly used both by people and ignorant programs. Our mailing lists serve as a permanent and concise record of our discussions, decisions, and operations, and we find it invaluable to be able to refer to individual messages in a simple and memorable way: "message 1210 in the calibration list", say. Other people can then easily jot that info down or directly find the message. Some message IDs even become shorthands for a particular topic or decision. We have also added trac InterWiki templates pointing into our mail archives (as listname:number), which encourages desirable cross-referencing (PRs, wiki pages, and SVN change logs can refer to mail messages, just as wiki pages could always refer to changesets and PRs, etc, etc.) But trac InterWiki templates can only interpolate $1,$2,... arguments into strings, and could not possibly calculate anything based on the _content_ of the messages. Globally unique IDs, hashed IDs, etc., are very appealing from various CS-y and techie points of view, but are simply not memorable to humans or knowable by dumb external programs. I think as much, or more, effort should be put into delivering a straightforwardly useable naming scheme as goes into making an arbitrary message recoverable from anywhere. Basically, "friendly URLs" should be a primary requirement, not an optional afterthought for careless geeks like me to get wrong later.... We long ago added an extremely simple ID handoff between MM 2.1.8 and pipermail, and though imperfect it has served us well. Basically, we hijacked the .post_id member in mailman (otherwise basically unused, and mysteriously a floating point number); CookHeaders stuffed it into a X-Mailman-Sequence-ID header line, and AfterDelivery incremented it. In turn, pipermail uses the header to feed a sequence ID into make_article, and the message is squirreled away as $mailinglist/all/%d.html. There are a few other minor matters (e.g. post_id was added to Decorators, a couple of templates were changed, we lost having 'ls' sort chronologically [did we have to add .last and .prev to the HyperDatabase classes?]), but it really was a minor bit of work. And for stability, as long as the archive files aren't lost, pipermail rebuilds should yield the same URLs even if junk messages have been deleted. [Oh, we did also add a "never rotate" policy to our archives, but that is finesseable. ] As an aside on other discussions, can you get away without using Message-ID or Date? I.e., aren't those just more of those tokens which were standardized back before the Internet got tricky enough to invalidate the standards? Mailing lists serialize incoming messages, and so can generate their own unique and trustworthy IDs. "UUIDs" would work, but if you can trust yourself to generate them, consecutive integers provide minimal, order-preserving, perfect hashing, too! Anyhow, we have found that people will enthusiastically refer by name to individual messages within mail archives if they can. - craig

3 2

--with-user and qmail-to-mailman.py
by Sebastian Wieseler Oct. 24, 2007

Oct. 24, 2007

Hello. If I use "--with-user" I got some errors with qmail-to-mailman.py. That's why it has got on line 67: local = re.sub("^mailman-","",local) But "mailman" isn't correct, because of "--with-user". Can you please fix that with something like: local = re.sub("^" + os.environ['USER'] + "-","",local) or even better re.escape(os.environ["USER"]) Lots of greetings Sebastian 'kickino' Wieseler, one of the Savannah admins, -- ,= ,-_-. =. /"\ ((_/)o o(\_)) \ / ASCII Ribbon Campaign `-'(. .)`-' && X against HTML e-mail \_/ / \

2 1

mta integration
by Gordon Messmer Oct. 17, 2007

Oct. 17, 2007

I'd like to write a filter for the Courier MTA which will run mailman's approval and spam tests before accepting messages to mailing lists. My filter framework is in python, so I think I can import the relevant bits of mailman (I'd like to start with MM2.1 compatibility). Can I get any pointers to the functions that check the sender for approval, and for the spam-specific posting filters?

2 2

lowercase in mailman create function
by Lawren Quigley-Jones Oct. 9, 2007

Oct. 9, 2007

I'm sorry if this issue was already addressed in later versions of mailman. I'm running debian etch and the mailman deb: dpkg -l | grep mailman ii mailman 2.1.9-7 Powerful, web-based mailing list manager I wrote a majordomo like interface for mailman which allows users to perform some backwards compatible functions via email, and access some information, like subscribed lists and owned lists via a web interface. When I released the interface to users, one of the first things a user did was to use all caps in the list they were attempting to create. My script verifies that the list doesn't already exist: if Utils.list_exists(listName): which it didn't and then passes it into the mailman Create function. mlist.Create(listName, listOwner, listPasswordHash ) There were no errors in the create process. The problem was that the list was actually a duplicate of an existing list which obviously wasn't in all caps. Since the process by which mailman gets the list configuration seems to be case insensitive, both lists would access the old config, so functionally I had two references to the same list. The solution to the problem was to just .lower() the listName string at the top of the list create function, but it might make sense to lower the string in the Create function as mailman is not case sensitive after that point. Let me know if you'd like me to open a bug for this issue... Thanks, -Lawren Quigley-Jones lawrenqj(a)gmail.com

5 6

running 2.1.9 completely from crontab
by Maximilian Mehnert Oct. 9, 2007

Oct. 9, 2007

I had a little problem because my provider does not allow running daemons. However, crontab is fine. Since setting up aliases does not work either and I did not want to change .procmailrc all the time, I'm using a catchall user and Maildir delivery. In order to make this work, I changed two things. diff -ur mailman-2.1.9/Mailman/Queue/MaildirRunner.py mailman-2.1.9-mod/Mailman/Queue/MaildirRunner.py --- mailman-2.1.9/Mailman/Queue/MaildirRunner.py 2005-08-27 01:40:17.000000000 +0000 +++ mailman-2.1.9-mod/Mailman/Queue/MaildirRunner.py 2007-10-06 16:08:29.000000000 +0000 @@ -123,7 +123,7 @@ # message was destined for. See verp_bounce() in # BounceRunner.py for why we do things this way. vals = [] - for header in ('delivered-to', 'envelope-to', 'apparently-to'): + for header in ('x-original-to','delivered-to', 'envelope-to', 'apparently-to'): vals.extend(msg.get_all(header, [])) for field in vals: to = parseaddr(field)[1] diff -ur mailman-2.1.9/bin/qrunner mailman-2.1.9-mod/bin/qrunner --- mailman-2.1.9/bin/qrunner 2006-01-19 01:07:40.000000000 +0000 +++ mailman-2.1.9-mod/bin/qrunner 2007-10-09 10:33:11.000000000 +0000 @@ -119,7 +119,8 @@ # Subclass to hack in the setting of the stop flag in _doperiodic() class Once(qrclass): def _doperiodic(self): - self.stop() + if self._oneloop()<1: + self.stop() qrunner = Once(slice, range) else: qrunner = qrclass(slice, range) First I added the x-original-to field since some mails got stuck. The second change in qrunner is the short hack to get all pending stuff done with one run from crontab: qrunner -v -r All -o Would that be something that could be included in future releases maybe as a separate option? Regards, Maximilian

1 0

The Approved: header in MM3
by Barry Warsaw Oct. 6, 2007

Oct. 6, 2007

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Those of you who have been watching the commit messages can see I've been making some good progress. I'm actually hoping to have a Mailman 3.0 alpha some time RSN which will almost allow you to run the system from the command line, but without a web u/i. So one of the things I'm looking at is the MM2.1 concept of an Approved header. If a message comes into a list with an Approved header (or an Approved line at the start of the message body), and that header has a password that matches the list admin or moderator password, the message is pre-approved and short-circuits the posting tests. The concept doesn't translate well in a Mailman 3 world where there is no shared admin or moderator password. Web access will be control via roles and protected by user authentication much like any modern web application. So the question is, what do we do about the Approved header? 1. We can drop the concept altogether. This means there'd be no way to post a message as coming from an approved source, with a bypass of the posting filters. Maybe because few people have MUAs that support adding custom headers, this feature just isn't used much in the real world these days. You'd still have the moderation bit for announce- only lists though. 2. Replace the concept with some other email authentication mechanism, e.g. something more secure like a signature check. The problem with this is that I still don't think message signing is common practice outside our small community of geeks. 3. Allow an owner or moderator to use their own password in the Approved header. I'm not crazy about this because it has to be sent in the clear and if (when?) it gets compromised, their account is compromised, and this includes their administration of the mailing list. 4. Add a new shared password just for this purpose. You'd still have to communicate it to all your moderators, probably via the web page, but at least this password wouldn't have any other purpose so if (when?) it gets compromised, the only asset it protects is approved postings. Bad yes, if a spammer gets it, but easily changed and hopefully fairly limited in the damage it can do. 5. Your suggestion. Comments? I think my preference would be for #1 with future support for #2 and just accepting the fact that message signatures are for power users. Maybe that set is pretty close to the set of people currently using Approved anyway. Cheers, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iD8DBQFHAwd62YZpQepbvXERAmMiAKCm3EyxA1CWxWyz4zWkzNwIDpCNKQCbBSXz hGqwpKEGmUScNjov68TUdgs= =gUiT -----END PGP SIGNATURE-----

7 6

Improving the archives
by Terri Oda Oct. 3, 2007

Oct. 3, 2007

Since I've largely finished up the coding contract that was eating up a lot of my time, I'm thinking that I'd like to do some coding for fun. And nothing says fun like trying to fix the Mailman archives! ;) I'm trying to remember all the things people have suggested for the archives in the past so I can figure out what needs to be done and what might be nice to have, and see if this is doable in the time I have in the foreseeable future. The big things people wanted most, if I recall correctly, included: - modernized HTML/CSS/Themes (preferably to match a modernized web interface... is that all set up now?) - archive links that won't break if the archive is rebuilt - better address obfuscation (maybe by generating pages through cgi) - search - not adding a billion dependencies to Mailman Here's the list from the wiki's Mailman 2.2 page: http:// wiki.list.org/display/DEV/Mailman+2.2 * Reconsider using a 3rd-party archiver * Perhaps URLs to messages should be based on message-ids instead of message numbers so that regenerating archives can't break links. This must include backward compatible links * Ditch direct access and vend all archive messages through CGI so that we can do address obfuscation, and message deletion, etc. on the fly (with caching of course, but have to worry about web crawlers). * Add RSS feed * Allow for admins to remove or edit messages through the web. * Move archive threads into another list? * Put archives in the list/mylist directory. * Add a search option * Make archives default template look and feel similar to Web UI (whatever it looks like after the Summer of Code project is done) * Make archive templatable (at least by changing CSS) so they can match people's existing site look-and-feel * MUAs usually make URLs clickable. An new Archive could be used when posts are distributed, in the footer, so that each message has a link to the whole thread in the Archive. * Present all messages in a thread at once, and offer plaintext download of the whole thread * Put messages into a database and/or move away from mbox as the canonical storage format. So the questions are: (1) Is anyone working on this already? (2) What else is on people's wish lists for a pipermail replacement? Terri

16 63