Fwd: Re: Mailing list membership.
Dear friends of Mailman,
in the IETF discussion list we have a discussion about the bounces, that are created based on the DMARC processing.
I know, from a discussion in this mailman list, that mailman follow strong the RFC 2821 (SMTP) and reject this DMARC nonsense.
I send you my answer to Theodore and Khaled in the IETF discussion list for verify. I have not so much experience with that. But i know from many mailman lists that they have always this bounce-errors.
The consequence should be, that the members in the list should change her mailbox servers to avoid this "bounce errors", based on DMARC processing, or not?
For me, i don't like any form of work arounds. We need a clear base. And this can only be the standard RFCs.
What you think about?
many greetings, willi
-------- Forwarded Message -------- Subject: Re: Mailing list membership. Date: Wed, 1 Mar 2017 16:29:01 -0300 From: willi uebelherr <willi.uebelherr@riseup.net> To: IETF discussion <ietf@ietf.org>
Dear Theodore, many thanks for your explanation to the RFC 2821 (SMTP). The answer from Khaled i have included, because it goes to me as a private message.
Maybe, based on my bad english, i feel some confusion with the terms.
Khaled, like i and you, is member of the IETF discussion list. This means, he receive all emails that are distributed over the list.
Khaled, like i and you, use the maillist server mailman from IETF discussion list to distribute his messages to all members in the list.
The IETF discussion list don't follow the DMARC processing. This means, it act only outside.
Khaled, like i and you, use a mail box server system as the interconnection point to the list. Khaled use hotmail, you use mit.edu, i use riseup.net.
This means, the actors are the mailbox servers with the mailman maillist server IETF discuss in both direction.
I understand and agree absolutly, that the maillist server never change the From-line in the header. He create the Return-Path-line and/or Error-To-line for error response from the receiver mailbox server system. The bounce-information.
The mailman maillist server use bounce-counters for every member and some limits for this bounce-counter. If the limit exceeds, and the admin-group do nothing, then the maillist server mailman disable the delivery. It is not an unsubscription.
The admin-group have to follow the incremental increase of the bounce-counters to understand, what is the background. Maybe, the mailbox is full, or don't exist or is the result of this stupid DMARC processing.
The DMARC processing is defined in the DNS info. But we can ignore it, or not? The admin-group can inform the member to change her mailbox server to "avoid more errors" like Khaled wrote. The IETF discussion admin-group can only inform about the error sources. The members have to change her mailbox servers, or not?
Based on that process, we can clean all this nonsense in our IETF lists environment and work strong based on the RFC 2821, like mailman do it.
What do you think about?
many greetings, willi
On 01/03/2017 07:50, Khaled Omar wrote:
Hi Willi,
Mailman never change the "From"-header. Therefore, the From-Header always points to the author of the email. What you think, is that the correct, compatible way? I think, yes.
Such case is out of our hands, other e-mail service providers are welcome to be used just if this will add a value and avoid more errors.
Best Regards, Khaled
On 01/03/2017 01:49, Theodore Ts'o wrote:
On Tue, Feb 28, 2017 at 05:29:24PM -0300, willi uebelherr wrote:
related to the problem, what Khaled explained, what is your proposal?
What are your "compatible with internet mailing lists" mail systems?
RFC 2821, Simple Mail Transfer Protocol, section 3.10.2
"To expand a list, the recipient mailer replaces the pseudo-mailbox address in the envelope with all of the expanded addresses. The return address in the envelope is changed so that all error messages generated by the final deliveries will be returned to a list administrator, not to the message originator, who generally has no control over the contents of the list and will typically find error messages annoying."
This is the SMTP Envelope From field. The FROM field is not changed, but the SMTP return address is changed, so that bounces go to the mailing list administrator as opposed to the person who sends mail to the mailing list.
Unfortunately, if you are using a system whose domain requests that all recipients enforce DMARC alignment, this specifically instructs recipients to bounce mail if the SMTP Envelope return address doesn't match the FROM field in the header. This means that they won't see mailing list mail as defined by the IETF Standards Track RFC 2821, which specifically says that is acceptable (and in fact a good thing) to change the SMTP envelope return address so that bounces (caused by people changing where they work, etc.) go to an administrator who can deal with them. But if the mailing list administrators gets too may bounces, and it's because the sending domain is requesting that mail be bounced, the only thing they can do is to unsubscribe the sender or the recipient.
Hence mailing list systems that enforce DMARC, or request DMARC processing, are fundamentally incompatible with mailing lists as defined by section 3.10.2 of RFC 2821.
If you want to participate in such mailing list, one of the best ways is to change to a mailing list system that doesn't do DMARC.
Best regards, -Ted
willi uebelherr writes:
Dear friends of Mailman,
Thanks for getting in touch!
in the IETF discussion list we have a discussion about the bounces, that are created based on the DMARC processing.
I know, from a discussion in this mailman list, that mailman follow strong the RFC 2821 (SMTP) and reject this DMARC nonsense.
RFC 2821 has little practical significance for DMARC. DMARC processing itself has very little to do with SMTP (except for the indirect relation through SPF -- more about that below). I have all the respect in the world for Ted T'so, but he seems to be not very familiar with DMARC, DKIM, and SPF.
Mailman can not and does not "reject" DMARC in any way (although some Mailman developers are pretty unhappy about it ;-). DMARC is a fact of life for mailing lists now.
First, the good news: I (as a representative of the GNU Mailman project, as well as for personal interest) have been participating in the DMARC discussions as well as in development of the new "Authenticated Received Chain" (ARC) protocol. ARC allows an intermediate site to cryptographically sign its authentication results, which (we hope) will allow end receivers to trust those for DMARC purposes. However widespread implementation is at least a year off (theoretically possible as the DMARC Consortium, Google, Yahoo!, and AOL are all participating in ARC discussions, but you know how the Internet moves).
Mailman 3 itself will be ARC-capable in limited ways in that time frame, and we expect Postfix, Exim, and Sendmail to implement it as well. (The "big guys" use MTAs developed in-house.)
Now, the bad news.
The consequence should be, that the members in the list should change her mailbox servers to avoid this "bounce errors", based on DMARC processing, or not?
From the list owner's point of view, "it would be nice if" posters would post from DMARC "p=none" domains. ("p=none" includes sites which have no DMARC DNS record at all; it is the implicit default that DMARC receivers must assume.) However, in the great majority of cases, users get very upset about being asked to use "p=none" mailboxes if that means changing their workflows. IETF mailing lists *may* be an exception, but I suspect there are a lot of people using Yahoo! and other DMARC-protected addresses in the IETF subscriber base.
For me, i don't like any form of work arounds. We need a clear base. And this can only be the standard RFCs.
Unfortunately, there are no RFCs that help here. DMARC is *intended* to prevent "unwanted" indirect mail flows, and unfortunately the heuristic chosen also will interdict mailing lists. What the DMARC RFC *should* say is "if your mailbox provider posts a DMARC policy other than 'p=none', that is automatically a policy of prohibiting posting to mailing lists". And that is precisely why the DMARC RFC is "Informative", rather than "Standards Track", as you would expect with an RFC with so much promise for reducing phishing and other mail abuse. The DMARC Consortium really wanted to avoid any language that would implicate DMARC users is negative effects of the standard, and an informative RFC allowed the Consortium members to retain control.
The only RFC-conforming option that allows you to avoid triggering DMARC backscatter is to turn off ALL message-modifying options: subject tags and serial numbers, and body headers and footers. (I believe that is currently the complete list of Mailman features that invalidate common DKIM signatures.)
Other than that, your choice is to turn your "p!=none" posters into backscatter bombs, or use of one of the options that Mailman provides to avoid triggering DMARC. There are more, and more attractive, options in the most recent Mailman 2 versions (at least 2.1.20), as Jim Popovich mentioned.
I hope this helps.
Below some comments on technical aspects of the appended correspondence.
- The IETF discussion list don't follow the DMARC processing. This means, it act only outside.
DMARC processing, by which I mean the protocols defined in RFC 7489, is in no way done by mailing lists. Anything that a mailing list or its MTA does to handle DMARC is a "workaround". (This will change with ARC, but ARC will provide no guarantees -- use of ARC is entirely optional for the receivers.)
I understand and agree absolutly, that the maillist server never change the From-line in the header.
Changing the From field in the RFC 2822 header is the most popular workaround by far. This is not a denial of your position, it's a statement of current common practice.
The mailman maillist server use bounce-counters for every member and some limits for this bounce-counter. If the limit exceeds, and the admin-group do nothing, then the maillist server mailman disable the delivery. It is not an unsubscription.
Under some setting of the bounce processing in Mailman, it *can* result in unsubscription.
The DMARC processing is defined in the DNS info. But we can ignore it, or not?
If you ignore it, your list(s) will be subject to adverse consequences based on DMARC processing at poster and subscriber addresses. That's your choice, of course.
Based on that process, we can clean all this nonsense in our IETF lists environment and work strong based on the RFC 2821, like mailman do it.
As Jim Popovich mentioned, the IETF lists are handled by GNU Mailman. According to the most recent message I received it is Mailman 2.1.18.
Ted T'so writes:
RFC 2821, Simple Mail Transfer Protocol, section 3.10.2
"To expand a list, the recipient mailer replaces the pseudo-mailbox address in the envelope with all of the expanded addresses. The return address in the envelope is changed so that all error messages generated by the final deliveries will be returned to a list administrator, not to the message originator, who generally has no control over the contents of the list and will typically find error messages annoying."
This is the SMTP Envelope From field. The FROM field is not changed, but the SMTP return address is changed, so that bounces go to the mailing list administrator as opposed to the person who sends mail to the mailing list.
This is true, but it is not part of the definition of mailing list, nor is there any popular mailing list software left that doesn't give you the option of fiddling with the FROM field (for several reasons such as anonymization, as well as working around DMARC).
Unfortunately, if you are using a system whose domain requests that all recipients enforce DMARC alignment, this specifically instructs recipients to bounce mail if the SMTP Envelope return address doesn't match the FROM field in the header.
This is a misunderstanding of the DMARC protocol. I won't go into details, but in practice the vast majority of originating domains use DKIM as well as, or instead of, SPF.[1] In the case of DKIM verification, the authorizing domain is the one claimed in the DKIM-Signature field, not the Envelope From.
Hence mailing list systems that enforce DMARC, or request DMARC processing, are fundamentally incompatible with mailing lists as defined by section 3.10.2 of RFC 2821.
This is false. In practice, a mailing list that does not alter the From, Subject, Date, and Message-ID header fields and does not alter the body is compatible with DMARC originators which provide DKIM signatures. That's pretty much all of them, as mentioned above.
As I mention above, Mailman can be configured that way, but it rarely is. A few list-owners run into legal issues where the list must provide visible legal disclaimers or unsubscription instructions, which typically occur in footers appended to the message body, invalidating the DKIM signature. I doubt such a legal requirement applies to the IETF lists, however.
Hope this helps.
Steve
Footnotes: [1] According to a source at DMARC Consortium, their analysis shows that nearly 100% (I forget the exact figure) of p=reject traffic is DKIM-signed. Of course that's heavily weighted by Yahoo! and AOL, the two biggest sources of p=reject traffic.
-- Associate Professor Division of Policy and Planning Science http://turnbull/sk.tsukuba.ac.jp/ Faculty of Systems and Information Email: turnbull@sk.tsukuba.ac.jp University of Tsukuba Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN
participants (3)
-
Jim Popovitch -
Stephen J. Turnbull -
willi uebelherr