Mailman 3 July 2013 - Mailman-Developers

Mailman 3.0 todo list
by Terri Oda 16 Jul '13

16 Jul '13

I've started a new "things left to do before Mailman 3 releases" list over at http://wiki.list.org/display/DEV/Mailman+3.0 Mostly this is so the next time someone asks me "So, when are you releasing?" I can at least say "We don't have a date yet, but here's the remaining todo list" but also because I think our wiki is rather information-poor on the subject of the current status of mailman 3 suite. I've populated the list with some todos leftover from PyCon, but I only really pulled out the postorius ones. Barry, Aurelian, maybe you could populate the Mailman and Hyperkitty lists? Terri

5 9

Adding DMARC support for Mailman 3
by Patrick Ben Koetter 14 Jul '13

14 Jul '13

Greetings, I am writing on behalf of a group of companies and single persons, who would like to see a limited feature set of the DMARC¹ standard supported by Mailman 3. Since I know we're all eager to get MM3 out as soon as possible and any additional new feature request stands against that I've contacted Barry offlist and asked if he'd agree that the companies involved pay us, sys4², to implement the feature. He did and we also agreed to dedicate a significant part of the payment to mailman's FSF donation account. Before we take out to write code, I would like to ask mailman-developers how it should be done to fit best into Mailman's architecture. Here are the DMARC features that should go into Mailman 3: - don't allow email that comes from a domain with a DMRAC record of p=reject - take ownership of the email and send it with a From: using the domain of the mailing list. (There's a patch for this for Mailman 2.1, which might might be helpful for Mailman 3.) - find the authentication-results header and rewrite it as an Original-Authentication-header: http://tools.ietf.org/html/draft-kucherawy-original-authres-00.html Speaking of an RFC written by Murray Kucherawy. I've contacted Murray in advance and asked him to assist in case we had any questions regarding his RFC(s). He subscribed and ready to help. I hope I was able to bring all parties required together to make a Mailman DMARC implementation come true and I am curious to hear what you have to say. p@rick ¹) DMARC <http://www.dmarc.org/> adds policies and more to DKIM, which gives digitally signed identity to sender domains and has become a cornerstone to reputation based mail management. Having DMARC support in Mailman 3 is - in my eyes - a reason for postmasters (not end-users) to upgrade their MM 2.x installation as soon as possible. ²) In case you ask yourself, who "sys4" is: We've accompanied Mailman 3 development since Pycon in Chicago (in 2008 ?). Florian Fuchs, whom I don't have to introduce, works with us as well Ralf Hildebrandt and I. We, Ralf and I, are on the python.org postmaster team. Also with sys4 are Antoine Nguyen and Christian Roesser - both are exerienced Python programmers and both have spent significant time developing email applications. -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein

4 7

Online Mailman Hackathon: July 14 0500-2300UTC
by Terri Oda 12 Jul '13

12 Jul '13

Hey all, We're going to do a virtual hackathon day for Mailman suite (Mailman/Postorius/Hyperkitty) on Saturday and you're all invited. I especially suggest that the GSoC students for both Mailman and Systers take advantage of this day and show up to ask questions and get some time with the mentors. Where: on IRC (#mailman on freenode) It's a virtual hackathon! When: Sunday July 14 0500 to 2300 UTC Who: anyone interested in Mailman dev As before, you're not expected to stay the whole time: people will drop in and out through that period as their time zone and other commitments permit. I, for example, will be on for a little bit at the beginning, then will be back around UTC 1600 until the end. We'll be doing a variety of things: bug triage and fixing, discussion of architecture, new feature development, helping each other with any blocking problems, spouting off crazy new ideas, code review and merging, etc. We're especially hoping to make sure we clear any issues we can relating to GSoC projects, but there's plenty of work to go around. New folk are welcome too. Hope to "see" you there! Terri PS - Here's a link to help you figure out what those times would be in your time zone. You can click "change cities" to add your own local time to the list: http://www.timeanddate.com/worldclock/meetingtime.html?iso=20130714&p1=394&…

1 0

Re: [Mailman-Developers] Adding DMARC support for Mailman 3
by Franck Martin 11 Jul '13

11 Jul '13

We are not asking mailman to do the work of DMARC here. There is openDMARC for that. On Jul 10, 2013, at 11:23 AM, Stephen J. Turnbull <stephen(a)xemacs.org> wrote: > Barry Warsaw writes: > >> For #1 you would have a rule that can answer the question of DMARC >> disposition. Rules output binary results, > > This is somewhat problematic. DMARC results are potentially > trivalent. If action is "reject" and pct is less than 100, some hits > are "rejects" and some are "quarantine". Misses are misses. So I > guess you do this with a chain of two rules, the first one verifying > the message and if that hits (ie, verification fails) the second one > rolls the dice for pct. > >> and if this rule hits, it would run an action, probably to discard >> the message, although it could also hold it or reject/bounce it. > > Silent discards without content analysis make me queasy. I guess we > can work around that by doing DMARC checks after the content checks, > although the draft implies the DMARC checks should be done early. Or > we could reject, but unfortunately we can't reject in the SMTP > transaction, so we need to issue a DSN. That makes me really queasy, > because DSNs for illegitimate mail suck all around. > > In case of a quarantine, maybe this should go into a separate queue > that silently waits for a moderator to look at the messages, and > discards them after a reasonable period of time (maybe two weeks?) So > they'd be there if somebody asks for a lost message, but otherwise no > bother.

2 1

Re: [Mailman-Developers] Adding DMARC support for Mailman 3
by Franck Martin 10 Jul '13

10 Jul '13

On Jul 9, 2013, at 8:19 PM, Barry Warsaw <barry(a)list.org> wrote: > On Jul 06, 2013, at 11:02 PM, Murray S. Kucherawy wrote: > >> >> (a) the second bullet above is a significant departure from current use (as >> I understand it), and fails the test of least surprise if we were going to >> suddenly see that MM3 does things quite differently than previous versions >> or, indeed, other packages; and > > Mailman has support for anonymous lists, which can rewrite the From header to > substitute the mailing list's posting address for the original author's > address. I frankly don't know how much that feature is used in MM2.1, if at > all. > This code was useful to do the patch for MM2.1. I too doubt it is much in use, after all we gave away our privacy long time ago on all the social networks and cloud services... but I guess it answered a need otherwise it would not have been there.

1 0

Re: [Mailman-Developers] Adding DMARC support for Mailman 3
by Franck Martin 10 Jul '13

10 Jul '13

On Jul 9, 2013, at 8:15 PM, Barry Warsaw <barry(a)list.org> wrote: > On Jul 02, 2013, at 12:44 AM, Patrick Ben Koetter wrote: > >> Before we take out to write code, I would like to ask mailman-developers how >> it should be done to fit best into Mailman's architecture. Here are the DMARC >> features that should go into Mailman 3: >> > > For #1 you would have a rule that can answer the question of DMARC > disposition. Rules output binary results, and if this rule hits, it would run > an action, probably to discard the message, although it could also hold it or > reject/bounce it. I doubt you'd want to accept the message. If the rule > misses then processing continues along as normal. Good idea, to held the message for approval. It is harder to know what you don't know... Holding the message for approval would give a chance for the list admin to guide the subscriber. > > Munging the From header is done by a processing handler, after the message has > already been accepted for posting, and is being prepared for delivery to the > list membership. Similarly, the Authentication-Results rewrite would be done > in the same or a different handler. With respect to both headers, I'm > assuming that the munging is the same for every recipient; there's a different > hook point for personalization of outgoing messages. > I thought about munging the From Header on a sender basis (if it has a DMARC policy) but that makes the list oscillating between two states therefore making life difficult for the recipient (rules to sort messages).

1 0

Re: [Mailman-Developers] Adding DMARC support for Mailman 3
by Franck Martin 10 Jul '13

10 Jul '13

----- Original Message ----- > From: "Stephen J. Turnbull" <stephen(a)xemacs.org> > To: "Franck Martin" <franck(a)peachymango.org> > Cc: "Mailman Developers" <Mailman-Developers(a)python.org> > Sent: Monday, July 8, 2013 2:01:43 AM > Subject: Re: [Mailman-Developers] Adding DMARC support for Mailman 3 > > Franck Martin writes: > > > If the From: contains the posting email of the mailing list, one > > would think that the default becomes reply to the list, but this is > > where Reply-To: can be used. > > Most users do not display Reply-To; many cannot (at least not at their > level of technical skill). This means that they get no indication of > the author of a message unless the author signs the body of the mail, > which often isn't done, and is impossible to enforce. So this setting > is simply unacceptable except on announce/advertising lists (where > Reply-To is usually set to some other address anyway) and on anonymous > lists (which as far as I know are relatively rare). > > If this option becomes a popular filter on large mail hosts, > discussion lists (ie, the kind of mailing list that Mailman was > originally intended to serve) will take a severe, perhaps fatal, blow. This is speculation, and broad fear of the future... The current practice for a postmaster is to trust (or not) emails from specific mailing lists, not who post them to the list. Adding DKIM to the list and taking ownership will only improve it. > > > I hope this helps alleviate concerns. > You should really read the code of the patch for MM2 and try it.

3 4

Re: [Mailman-Developers] Adding DMARC support for Mailman 3
by Franck Martin 10 Jul '13

10 Jul '13

Greetings, Patrick asked me to introduce a bit why DMARC and mailman. In one year DMARC has gained good support (60% of worldwide mailboxes are protected with DMARC http://www.dmarc.org/news/press_release_20130206.html), but like others I'm worried about the long tail. This is the reason some of the people working with DMARC.org have been sponsoring the openDMARC implementation to make it available on a large set of mail servers (cf http://www.trusteddomain.org/opendmarc/ for a list of sponsors). Some openDMARC packages are now available and I expect to see them as part of GNU/Linux distros anytime soon. Similarly, I'm interested to offer the option to list administrators to transition to a behavior that makes the lists safe/working/compatible with DMARC. As Patrick explained, there are about 3 possibilities, while I'm interested more in some than others (I personally experimented with the patch to mailman 2.1), it is only fair to offer the 3 options and let the list administrator choose the one more suitable for his/her needs. Once Patrick has a better understanding on how to best implement these 3 options, it will be easy, like for openDMARC, to sponsor the work to make it as part of mailman3. I know that several DMARC.org members have shown interest to do so. In an other year, with the help of the mailman community, we can progress more in the fight against fake emails. While this may sound like a sales pitch, there has not been so much excitement in email for a long time. Franck Martin https://www.linkedin.com/in/franckmartin ----- Original Message ----- From: "Patrick Ben Koetter" <p(a)sys4.de> To: "Mailman Developers" <Mailman-Developers(a)python.org> Sent: Monday, July 1, 2013 3:44:15 PM Subject: [Mailman-Developers] Adding DMARC support for Mailman 3 Greetings, I am writing on behalf of a group of companies and single persons, who would like to see a limited feature set of the DMARC¹ standard supported by Mailman 3. Since I know we're all eager to get MM3 out as soon as possible and any additional new feature request stands against that I've contacted Barry offlist and asked if he'd agree that the companies involved pay us, sys4², to implement the feature. He did and we also agreed to dedicate a significant part of the payment to mailman's FSF donation account. Before we take out to write code, I would like to ask mailman-developers how it should be done to fit best into Mailman's architecture. Here are the DMARC features that should go into Mailman 3: - don't allow email that comes from a domain with a DMRAC record of p=reject - take ownership of the email and send it with a From: using the domain of the mailing list. (There's a patch for this for Mailman 2.1, which might might be helpful for Mailman 3.) - find the authentication-results header and rewrite it as an Original-Authentication-header: http://tools.ietf.org/html/draft-kucherawy-original-authres-00.html Speaking of an RFC written by Murray Kucherawy. I've contacted Murray in advance and asked him to assist in case we had any questions regarding his RFC(s). He subscribed and ready to help. I hope I was able to bring all parties required together to make a Mailman DMARC implementation come true and I am curious to hear what you have to say. p@rick

2 1

Re: [Mailman-Developers] Adding DMARC support for Mailman 3
by Franck Martin 10 Jul '13

10 Jul '13

Yes, these are options and they should be off. It is important when you introduce options, you do not change past behavior automatically. 1) may not be necessary, if mailman recognizes the bounce message as in section http://tools.ietf.org/html/draft-kucherawy-dmarc-base-00#section-15.8 eg "550 5.7.1 Email rejected per DMARC policy for example.com" and does not increase the unsubscribe/bounce counter for the receiving email address. I suppose MM3 bounce processing is better than with MM2, so this may be already addressed. Some people have requested this feature, so it is fair to include it, rather than them having to tweak the associated MTA (which some do not have control). 2) mailman has one option: reply to poster or reply to list. I think the first one is default, but as mentioned, many list admins, enable reply to list. I won't enter in the details, but my experience is that reply to list increases participation with less technically savvy people and I read why people should not do that (but they do)... Anyhow, the option is there, so I prefer not to choose on the behalf of the List Admin, and it is best to not change this setting when the list Admins enables 2) If the From: contains the posting email of the mailing list, one would think that the default becomes reply to the list, but this is where Reply-To: can be used. If the list is set to reply to poster, the email address of the poster is added to the reply-to header If the list is set to reply to the list, then the email address of the list plus the email address of the poster is added to the reply-to header. The RFC allows as many email addresses there as needed. So if no reply-to header is present, otherwise the correct email addresses are added to the list. When you click the reply button in your MUA, you achieve same results when option 2) is on or off. Reply to all button, behaves the same in all cases. So no munging is done, unless the List is already set for munging and the reply to will contain the original author of the email retransmitted by MM3. 3) This draft has been on the table for a while, as Murray points, one large mailbox provider uses it in a proprietary way, but similar to what is in Murray's draft. So there is experience, and as far as I know they still do not think it is a bad idea. Nevertheless, I think mailman should not do the email authentication part, but be able to recognize "true" authentication-result headers coming from the MTA mailman uses and be able to rewrite them as an OAR. This keep the logic simple, and should be enabled if the MTA can control Authentication-Results headers and remove spoofed ones. Personally I think 3) is more complicated than 2) to put in place correctly as it requires tight configuration between MM3 and the MTA. I hope this helps alleviate concerns. PS: Stephen, Murray does not work for Google, and therefore cannot change the way gmail works. What you describe is a gmail "feature". ----- Original Message ----- From: "Stephen J. Turnbull" <stephen(a)xemacs.org> To: "Murray S. Kucherawy" <superuser(a)gmail.com> Cc: "Mailman Developers" <Mailman-Developers(a)python.org> Sent: Sunday, July 7, 2013 7:56:15 AM Subject: Re: [Mailman-Developers] Adding DMARC support for Mailman 3 Murray S. Kucherawy <superuser(a)gmail.com> writes: Hey, I've been trying to find the superuser at Gmail, and it turns out I've known him all along! Would you please fix the breakage where copies of one's posts aren't delivered to yourself, which is a real PITA when trying to diagnose Gmail problems?<wink/> (BTW, did you try to sign up as root? or Administrator? ;-) > I'm all for experimentation and being adaptive as new things come along, > and I'm obviously supportive of the DMARC effort. That said, I hope that > these are going to be configurable options, defaulted "off" for backward > compatibility. I second your concerns in principle, but I don't think you need to worry about these features being anything but optional. I suspect that a majority of our user don't actually have the fine control over their DNS needed to implement DMARC. > (a) the second bullet above is a significant departure from current > use Violating the RFC by munging From (for heaven's sake! have we learned nothing from the sad history of Reply-To munging?) is simply not acceptable for most purposes, although perfectly acceptable for advertising. ;-)

4 4

Re: [Mailman-Developers] Adding DMARC support for Mailman 3
by Barry Warsaw 10 Jul '13

10 Jul '13

On Jul 06, 2013, at 11:02 PM, Murray S. Kucherawy wrote: >I'm all for experimentation and being adaptive as new things come along, >and I'm obviously supportive of the DMARC effort. That said, I hope that >these are going to be configurable options, defaulted "off" for backward >compatibility. This is because: Current MM3 doesn't have a great way to expose plugin configuration options to the REST API, which is what a web ui would access to control this stuff. I've thought about it a lot, so I think I know how I'd do it, but there's details to be worked out and code to be written around this. >(a) the second bullet above is a significant departure from current use (as >I understand it), and fails the test of least surprise if we were going to >suddenly see that MM3 does things quite differently than previous versions >or, indeed, other packages; and Mailman has support for anonymous lists, which can rewrite the From header to substitute the mailing list's posting address for the original author's address. I frankly don't know how much that feature is used in MM2.1, if at all. -Barry

1 0