Fixing DMARC problems with .invalid munge
I realize I'm a bit late to this party, but this is a technique that I don't think has been addressed here. On my lists I've fixed the DMARC bounces by rewriting From: lines of DMARC'ed domains like this on the way out:
From: Marissa <mmeyer@yahoo.com>
to
From: Marissa <mmeyer@yahoo.com.invalid>
Before you tell me I'm nuts, hear me out. I've actually implemented this, and it works.
Advantages:
Really fixes DMARC problems
Everyone can still post to lists
Doesn't change the way users in non-DMARC'ed domains work
Leaves the author's address in the headers, so you can reply and use MUA sorting rules, give or take usenet-style unmunging. (This is a reason we dislike putting the list address in the From: line.)
Code is simple
Gives people a hint of where the actual problem lies
R's, John
PS: I use .invalid because RFC 6761 promises it will never be allocated to anyone.
On 05/04/2014 06:26 AM, John Levine wrote:
I realize I'm a bit late to this party, but this is a technique that I don't think has been addressed here. On my lists I've fixed the DMARC bounces by rewriting From: lines of DMARC'ed domains like this on the way out:
From: Marissa <mmeyer@yahoo.com>
to
From: Marissa <mmeyer@yahoo.com.invalid>
It's been mentioned on mailman-users.
See the post at <https://mail.python.org/pipermail/mailman-users/2014-May/076761.html> and the two following replies. It came too late for 2.1.18, but we would be interested in your experience for a follow-up release.
Our concerns are the possibility of mail being rejected by recipient MTAs because of the invalid From: address and user complaints about difficulty in replying to the poster.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
From: Marissa <mmeyer@yahoo.com.invalid>
Our concerns are the possibility of mail being rejected by recipient MTAs because of the invalid From: address and user complaints about difficulty in replying to the poster.
Those are exactly the things I was worried about, too.
I've seen no rejections at all due to the address. In an earlier iteration I tried using a null group address like this, which got rejected all over the place:
From: "Marissa <mmeyer@yahoo.com>" :;
(That is legal syntax since March 2013 per RFC 6854, but few MTAs have been updated to take note, and anything that does DMARC processing tends to freak out if there is not exactly one From: address.)
The .invalid hack seems fine, no bounces, and no complaints about disappearing mail. There are mutant versions of this hack where you append a name with a wildcard that resolves but has an MTA that rejects all the mail, and a really evil one where you append a name that points to a server that rewrites the address and remails it, e.g. mmeyer@yahoo.com.remail.lists.org -> mmeyer@yahoo.com.
For replies, I expected complaints, since I'm using it on some busy lists for my church where people complain about every little burp, but to my surprise I've gotten none. I think one reason is that you can still use an unmunged Reply-To, which a lot of users do, and the other is that it's pretty obvious what to do to get the address to work, unlike trying to guess the author's address if the From: is the list.
Something I have considered but not implemented is to add a fake Cc: line with the unmunged address so reply-to-all will work. It's not clear whether that would be more confusing than useful.
Regards, John Levine, johnl@taugh.com, Taughannock Networks, Trumansburg NY Please consider the environment before reading this e-mail.
Hi John, At 08:16 04-05-2014, John Levine wrote:
The .invalid hack seems fine, no bounces, and no complaints about disappearing mail. There are mutant versions of this hack where you append a name with a wildcard that resolves but has an MTA that rejects all the mail, and a really evil one where you append a name that points to a server that rewrites the address and remails it, e.g. mmeyer@yahoo.com.remail.lists.org -> mmeyer@yahoo.com.
For replies, I expected complaints, since I'm using it on some busy lists for my church where people complain about every little burp, but to my surprise I've gotten none. I think one reason is that you can still use an unmunged Reply-To, which a lot of users do, and the other is that it's pretty obvious what to do to get the address to work, unlike trying to guess the author's address if the From: is the list.
Was there any occurrence of the ".invalid" in replies which were posted to the mailing list [1]?
Regards, -sm
- I assume that it would be caught on message submission. I am asking the question as what happens in practice might be different.
John Levine writes:
Before you tell me I'm nuts, hear me out. I've actually implemented this, and it works.
You're not nuts. However, your definition of "works" is necessarily limited to what you personally can see, in only a couple of weeks. It does *not* take into account the potential damage to interoperability over time of this hack becoming wide-spread.
Advantages:
- Really fixes DMARC problems
That's a matter of opinion. The DMARC-using domains will disagree, I think, as it still means that you are "impersonating" their users (see below), and making DMARC ineffective as a means of reducing spam and phishing. But we'll see about that soon enough.
Disadvantages:
The big one is that we can only guess what the long-run consequences are going to be. In particular, I think in the short-run we can definitely foresee (1) spreading invalid addresses, which will bounce if automatically inserted as destination addresses, all over the net, possibly creating more backscatter, and (2) a tendency for people to see "@yahoo.com.invalid" as a *valid* form of Yahoo! address ("computers are so weird"). Tools, most of them simple, will be created to hide and to strip ".invalid" automatically, meaning you can hide any arbitrary domain from DMARC behind ".invalid". Spammers and phishers will pick up on this, with an eventual adverse effect on your lists' ability to deliver mail (not to mention whatever responsibility we have for any victims we have trained to ignore ".invalid").
Note that these effects, if operational, tend to hurt everybody on the net. Removing yahoo.com, and other domains with "p=reject" policies, entirely from "From" hurts Yahoo! users and people who want to communicate with Yahoo! users, but really, the bad effects will stop there, I think. Yahoo! gets what conspiracy theorists (Hi, Lindsay!) think they want, namely locking their users into Yahoo! groups, but I think they're going to find that the opposite effect (of annoying their users so they move to a more flexible domain) is more important, and DMARC will fall of its own weight without leaving too much of a mark on the rest of the mail system.
OTOH, the natural response of the DMARC folks is going to be to try to control this. I can't imagine, if they succeed in agreeing on how to do so, that it will cause less damage to interoperability than their current behavior.
My bottom line is that I would prefer not teaching Mailman to violate RFC 5322 at all (yes, I know that ship has long since sailed), and if we must we should limit it to what the "owners" of those addresses recommend.
Remember, the DMARC people are by and large employed by companies with *no vested interest* in the continued success of the email system, especially for forums: in most cases they'd be happier if mailing lists (and newsgroups) went away and their users were locked into their web forums. We, on the other hand, depend very much on the smooth working of the email system -- the only effect of fighting fire with fire is to increase the damage *we* suffer from first- to second-degree burns.
If third parties' experiments with .invalid become overwhelmingly popular, then practically speaking Mailman will cave on an option to munge From by appending ".invalid" the same way it had to cave on Reply-To munging. But let's not lead the charge of the Light Brigade.
Regards, Steve
- Really fixes DMARC problems
That's a matter of opinion. The DMARC-using domains will disagree, I think, as it still means that you are "impersonating" their users (see below), and making DMARC ineffective as a means of reducing spam and phishing. But we'll see about that soon enough.
No, I can see that mail using this hack is delivered into my AOL and Yahoo accounts. I know a people at AOL and Yahoo, and they quietly admit that screwing up mailing lists was not the goal, it's a side effect of a clumsy attempt to clean up after security breaches. See my blog entry at http://jl.ly/Email/aoldmarc.html. The conspiracy theories about Yahoogroups are just that.
Note that these effects, if operational, tend to hurt everybody on the net. Removing yahoo.com, and other domains with "p=reject" policies, entirely from "From" hurts Yahoo! users and people who want to communicate with Yahoo! users, but really, the bad effects will stop there, I think.
The DMARC cartel includes the largest mailbox providers in the world. There may be pockets of the net where you can thumb your nose at them, but in the world I live in, people depend on the lists I run, and it is simply not an option to tell all the Yahoo and AOL users to go away, much though we might think they deserve it.
By the way, in the long run, the plan is to persuade the DMARC crowd to mitigate the damage themselves, if not by dropping inappropriate DMARC policies, by figuring out how to whitelist the 30,000 (Yahoo's count) list and other providers that they've screwed up. One advantage of this hack is that you can just turn it off when you don't need it, much easier than the stuff that puts the list address in the From: line which affects everyone.
Regards, John Levine, johnl@taugh.com, Taughannock Networks, Trumansburg NY Please consider the environment before reading this e-mail.
John R Levine writes:
One advantage of this hack is that you can just turn it off when you don't need it, much easier than the stuff that puts the list address in the From: line which affects everyone.
You're wrong on both counts. In Mailman 2.1.18, "From" munging is equally easy to turn off -- two clicks on an admin page. In Mailman 2.1, it's unlikely that anything will ever get easier than that (not a lot of Web 2.0 fancy click-once-and-be-done-with-it stuff in 2.1). And Mailman 2.1.18 has an option that munges "From" only in case of a published "p=reject" DNS record.
As for the rest of your post, you didn't say anything new and you didn't rebut my point about the limitations on what will be visible to individual postmasters in the short run. Until we have more evidence (or at least much better theory than I've seen so far, including from myself) on the long-run system-wide effects of ".invalid", I'm going to oppose implementing it *in Mailman*.
To give you one example that demonstrates how little we know about system-wide effects, I don't recall predictions of mass unsubscribes of third parties when a major freemail provider changed policy to "p=reject". That's an "oops". I doubt that ".invalid" will have that obvious an effect in the short run, but I think there is a real, worrisome possibility that it will have a more subtle, infrequent effect that cumulates over time.
Note that I am not telling *you* what to do or not to do. I am saying that the *Mailman Project* should act according to "Look Before You Leap" (and IMO preferably not leap ever, but that's just my opinion), rather than being seduced by "Easier to Ask Forgiveness than Permission." It may be *easy* for Mailman to ask, but if things do go south, forgiveness may not be forthcoming.
One advantage of this hack is that you can just turn it off when you don't need it, much easier than the stuff that puts the list address in the From: line which affects everyone.
You're wrong on both counts. In Mailman 2.1.18, "From" munging is equally easy to turn off -- two clicks on an admin page.
You're missing the point. If you configure a list to put the list's address in the From: line, users have to change they way they work for all mail from the list, and if you change it back, the users have to change again.
Regards, John Levine, johnl@taugh.com, Taughannock Networks, Trumansburg NY Please consider the environment before reading this e-mail.
John R Levine writes:
One advantage of this hack is that you can just turn it off when you don't need it, much easier than the stuff that puts the list address in the From: line which affects everyone.
You're wrong on both counts. In Mailman 2.1.18, "From" munging is equally easy to turn off -- two clicks on an admin page.
You're missing the point.
No, I just know that "From" munging in Mailman[1] is designed to avoid the problem you're talking about. See below.
If you configure a list to put the list's address in the From: line, users have to change they way they work for all mail from the list,
It won't be "all mail" in the great majority of cases, I believe, as the preferred (perhaps eventually default) configuration will be dmarc_moderation_action = wrap_message (or mung_from).
and if you change it back, the users have to change again.
I suspect you misunderstand both the results of your experiment and how "wrap_message" and "mung_from" actually work. That is, the reason you've had no complaints from your users about ".invalid" is that they very rarely reply to author anyway. From your description of the lists you serve I would expect that users rarely feel a need for a personal reply (as opposed to a public reply directed to a person). I could be wrong, but how do you get data to show I'm wrong?
In any case, here are the reply-to-author algorithms for each form of
From munging:
.invalid in From, non-Reply-To munging list:
1a. Hit "r" (or click button labeled "Reply"), as usual.
2a. Mark and delete ".invalid", different from pre-DMARC.
3a. Compose and send message, as usual..invalid in From, Reply-To munging list:
1b. Hit "r" (or click button labeled "Reply"), bs usual.
2b1. Mark author address except ".invalid", different.
2b2. Copy it.
2b3. Delete or mark list-post.
2b4. Paste author address.
3b. Compose and send message, as usual.list-post in From, author in Reply-To (non-Reply-To-munging):
1c. Hit "r" (or click button labeled "Reply"), as usual.
2c. No-op, as usual.
3c. Compose and send message, as usual.list-post in From, author and list-post in Reply-To:
1d. Hit "r" (or click button labeled "Reply"), as usual.
2d. Mark and delete <list-post>, as usual for Reply-To-munging
lists which add author to Reply-To. In GMail this is
actually easier than marking, as you can click on a
delete-address button.
3d. Compose and send message, as usual.There is one case that may be different, which is a "replace Reply-To with list-post" Reply-To-munging list, where IIUC the user's identity will be in the display name as "A. User via list-post" but her address may not be. If so, I think the answer is "Don't do that" if the list- owner cares about reply-to-author, or perhaps Mailman's treatment can be improved. Note that the ugly display name in "From" should never be automatically used by a "reply" function, because "Reply-To" will always be present, and that overrides "From". I believe that all major MUAs conform.
A similar analysis can be done for reply-to-list, with I believe similar results (the dog wants to be walked, gotta go).
I see no real difference between 2a and 2d, and I doubt users will be particularly unhappy with switching from 2b to 2d. It's true that users will have to learn to trust that author is indeed in reply-to (most MUAs don't display it), but I think they will, quite quickly.
For wrap_message, there is *no change at all* once you're actually reading the message, but you probably have to take some action to read the encapsulated message.
Footnotes: [1] At least, is intended to work. Mailman 2.1.16 and 2.1.17 had some problems but I believe 2.1.18 now works as intended.
Hi,
This is my first post here so please be gentle. I know I am risking telling certain people how to suck eggs.
I'm not currently a Mailman user but will probably begin to use it soon. I've followed the Mailman-Developers list for some time to familiarise myself with how Mailman works.
Apologies also for the long message.
On 04/05/2014 19:08, Stephen J. Turnbull wrote:
Advantages:
- Really fixes DMARC problems
That's a matter of opinion. The DMARC-using domains will disagree, I think, as it still means that you are "impersonating" their users (see below), and making DMARC ineffective as a means of reducing spam and phishing. But we'll see about that soon enough.
Surely that is a case of "so be it". DMARC (well, to be precise, Yahoo's particular implementation of it combined with Yahoo's size in the market) has broken longstanding and valid usage patterns and, for simple practical purposes, needs to be routed around in a way that still retains interoperability with Yahoo.
I do not think that this method of working around Yahoo's DMARC implementation will necessarily impact the usefulness of DMARC as an impersonated-spam/spoofing reduction tool. Whilst it can be argued that adding ".INVALID" (or something similar, see also below) is still impersonating Yahoo users, it can also be argued (more persuasively in my opinion) that "yahoo.com.INVALID" or "yahoo.com.REMOVEME" or similar is quite clearly not impersonating any valid Yahoo user *when sent from a mail list*.
Disadvantages:
The big one is that we can only guess what the long-run consequences are going to be. In particular, I think in the short-run we can definitely foresee (1) spreading invalid addresses, which will bounce if automatically inserted as destination addresses, all over the net, possibly creating more backscatter
But who is going to be emailing using those addresses? Surely the only people who would be emailing from such addresses for the purposes of spoofing are spammers and any additional backscatter from such a source would surely only be an incremental additional to the problem of spamming. DMARC is, in any case, intended to reduce such backscatter.
and (2) a tendency for people to see "@yahoo.com.invalid" as a *valid* form of Yahoo! address ("computers are so weird").
To my mind this is a valid criticism of the choice of words (i.e. ".invalid" can quickly start to be taken to be valid!) but not of the idea in general.
Perhaps a better form of words can be found. For example:
.REMOVEME .ThisMailProvidersDMARCImplementationIsBroken .RemoveThisToEmail .MungedForDMARCPleaseRemoveToEmail
In other words, something that non-technical end users will not so easily become inured to.
Tools, most of them simple, will be created to hide and to strip ".invalid" automatically, meaning you can hide any arbitrary domain from DMARC behind ".invalid". Spammers and phishers will pick up on this
This is surely going to happen anyway if spammers want to harvest mail lists as spam target sources or to harvest identities for spoofing (which, as I understand it, is what DMARC is particularly intended to prevent).
I don't see why it should affect mail lists and I don't see why it should prevent DMARC from working to prevent impersonated/spoofed spam. Indeed, it is the *removal* of suffixes like ".INVALID" (or whatever) that DMARC is intended to eventually make worthless (not the addition of them).
Yahoo! gets what conspiracy theorists (Hi, Lindsay!) think they want, namely locking their users into Yahoo! groups, but I think they're going to find that the opposite effect (of annoying their users so they move to a more flexible domain) is more important, and DMARC will fall of its own weight without leaving too much of a mark on the rest of the mail system.
Sadly I suspect that the vast majority Yahoo users will be entirely oblivious to it.
Out of the very few (relatively speaking) Yahoo users who use services which are impacted by Yahoo's p=reject DMARC implementation many will blame the services themselves. The remaining few who understand the issue will move away to better providers, but I suspect this will be a negligible proportion of Yahoo's user base.
As far as I can see, DMARC is set to work as Yahoo and its other backers intend for it. They have the market size to push it through regardless of standards.
OTOH, the natural response of the DMARC folks is going to be to try to control this. I can't imagine, if they succeed in agreeing on how to do so, that it will cause less damage to interoperability than their current behavior.
But do they actually need to? As I understand it, they wish to use DMARC to prevent spam that impersonates or spoofs their users or clients. Spammers who strip off munged email suffixes and spoof-send using the unmunged email address will indeed be blocked by DMARC.
By munging email addresses we are surely going along with the aim of not meaningfully impersonating users of DMARC-using mail providers. We are making it clear who the original sender is but in a form that does not make it look like the email was sent by a third party spoofing the original sender in question.
and if we must we should limit it to what the "owners" of those addresses recommend.
The recommendations that dmarc.org makes for mailing list providers at http://dmarc.org/faq.html#s_3 could be helpful in some cases but do not address the full issue that adding a suffix is intended to work around (and it is an issue which *needs* be to worked around somehow). That said, adding a suffix is simply a variation of the suggestion in section 3 of the linked FAQ.
The reference in the FAQ to Original Authentication Results at http://tools.ietf.org/html/draft-kucherawy-original-authres-00 looks rather interesting too.
Remember, the DMARC people are by and large employed by companies with *no vested interest* in the continued success of the email system, especially for forums: in most cases they'd be happier if mailing lists (and newsgroups) went away and their users were locked into their web forums. We, on the other hand, depend very much on the smooth working of the email system -- the only effect of fighting fire with fire is to increase the damage *we* suffer from first- to second-degree burns.
And yet we need somehow to route around the problem they have caused us. Adding suffixes seems like a kludge but a not unreasonable one and it is also one that should not negatively impact DMARC or users of DMARC, as far as I can see.
If third parties' experiments with .invalid become overwhelmingly popular, then practically speaking Mailman will cave on an option to munge From by appending ".invalid" the same way it had to cave on Reply-To munging.
Would this really be a problem? If it become a de facto standard the all well and good (but let's get the wording right). Mail providers such as Yahoo could even play along with it by recognising it and it would not negatively impact the effectiveness of DMARC from their perspective or spam filtering, from what I can see.
-- Mark Rousell
PGP public key: http://www.signal100.com/markr/pgp Key ID: C9C5C162
Mark Rousell writes:
I do not think that this method of working around Yahoo's DMARC implementation will necessarily
Nor do I. I point to the *possibility* and our lack of ability to predict effects. The RFCs have proven over time to give us a system that works smoothly. We have rules of thumb that help to understand why they work as well as they do, but the most important one is that RFCs must be shown to work in practice before they become Proposed Standards. Ie, don't expect something to work until you see it.
it can also be argued (more persuasively in my opinion) that "yahoo.com.INVALID" or "yahoo.com.REMOVEME" or similar is quite clearly not impersonating any valid Yahoo user *when sent from a mail list*.
Ah, but what matters here is not your opinion or mine, nor even those of the DMARC folks or the sysadmins at Yahoo. What matters is what "just plain folks" think. Remember, according to AOL, 2% of such users were suckered by the recent phishing attack at AOL, and that presumably translates to around 50,000 users! If 2% of those 2% adopt the interpretation I fear they will, that's 1000 users. I'm not willing to allow Mailman to be (partly) responsible for that without trying to find a much better way first.
But do they actually need to? As I understand it, they wish to use DMARC to prevent spam that impersonates or spoofs their users or clients. Spammers who strip off munged email suffixes and spoof-send using the unmunged email address will indeed be blocked by DMARC.
No, that's the whole point. They will *not* strip the suffix, and instead prefix the phishing attack with
We have repeatedly attempted to reach your email address, but our
mail has been rejected due to your ISP's DMARC configuration.
Thus we have used the .invalid convention to work around this
problem for this important message.You saw it here first! And don't think that the phishers are too dumb to figure it out for themselves. N.B. This trick doesn't work if the list-post address is in From.
The recommendations that dmarc.org makes for mailing list providers at http://dmarc.org/faq.html#s_3 could be helpful in some cases but do not address the full issue that adding a suffix is intended to work around (and it is an issue which *needs* be to worked around somehow).
The wrap_message feature implemented in Mailman 2.1.18 is such a workaround. It completely takes "ownership" (aargh!) of the message in a way which is 100% RFC-conformant and causes no loss of information.
The main problem is that it is known to be inconvenient for users of at least some versions of the Apple MUA on iPhones (in general, for anybody whose MUA can't handle MIME digests well).
Adding suffixes seems like a kludge but a not unreasonable one and it is also one that should not negatively impact DMARC or users of DMARC, as far as I can see.
That is not good enough for me. When you can say, "here's a $10,000 prize for the first report of a serious negative effect of this kludge, and it's in escrow with a reputable service", then I'll consider this kind of argument. $10,000 is a quite reasonable criterion, given that if you're wrong, you may be causing $1 worth of annoyance to each of 500 million people.
Hi Stephen, At 01:07 05-05-2014, Stephen J. Turnbull wrote:
Nor do I. I point to the *possibility* and our lack of ability to predict effects. The RFCs have proven over time to give us a system that works smoothly. We have rules of thumb that help to understand why they work as well as they do, but the most important one is that RFCs must be shown to work in practice before they become Proposed Standards. Ie, don't expect something to work until you see it.
This is a nit. There isn't any requirement that RFCs have to be shown to work in practice before they become Proposed Standards.
of the DMARC folks or the sysadmins at Yahoo. What matters is what "just plain folks" think. Remember, according to AOL, 2% of such
Yes.
No, that's the whole point. They will *not* strip the suffix, and instead prefix the phishing attack with
We have repeatedly attempted to reach your email address, but our mail has been rejected due to your ISP's DMARC configuration. Thus we have used the .invalid convention to work around this problem for this important message.
Yes.
Regards, -sm
SM writes:
RFCs must be shown to work in practice before they become Proposed Standards. Ie, don't expect something to work until you see it.
This is a nit. There isn't any requirement that RFCs have to be shown to work in practice before they become Proposed Standards.
Don't you have that backwards? It's pointing out lack of a formal hard requirement that is nit-picking. After all, Postel's Principle isn't written in any IETF procedure manual. Would you call that one a "nit", too?
Hi Stephen, At 23:28 05-05-2014, Stephen J. Turnbull wrote:
Don't you have that backwards? It's pointing out lack of a formal hard requirement that is nit-picking. After all, Postel's Principle isn't written in any IETF procedure manual. Would you call that one a "nit", too?
I labelled my previous comment as a nit (the previous comment was unimportant). I would not describe anything someone else writes as a nit.
Regards, -sm
SM writes:
Hi Stephen, At 23:28 05-05-2014, Stephen J. Turnbull wrote:
Don't you have that backwards? It's pointing out lack of a formal hard requirement that is nit-picking. After all, Postel's Principle isn't written in any IETF procedure manual. Would you call that one a "nit", too?
I labelled my previous comment as a nit (the previous comment was unimportant). I would not describe anything someone else writes as a nit.
OK, thanks for the apology. And sorry on my part for assuming you were calling the (informal) rule a nit.
We have repeatedly attempted to reach your email address, but our mail has been rejected due to your ISP's DMARC configuration. Thus we have used the .invalid convention to work around this problem for this important message.
They already do about a million similar things. Among the various attacks that DMARC does not address are these:
From: Paypal Security <boris@rbn.ru> From: Paypal Security <security@paypaI.com>
(Keep in mind that many MUAs, probably a majority, don't show the address at all, only the comment.)
I wouldn't waste time worrying about whether various hacks might make it 0.0001% easier to phish people.
R's, John
John Levine writes:
I wouldn't waste time worrying about whether various hacks might make it 0.0001% easier to phish people.
Will you please stop focusing on *your* logic, and start thinking about what happens if people with different interpretations of the facts take action on those interpretations?
*I* am not really worried about 0.0001% easier to phish (although I think my "2%" is a more accurate estimate). What I worry about is "what if Yahoo! and AOL think ...". We already know that they think differently from us. They are desperate and grasping at straws, as far as I can see. The whole SPF-ADDoS-DKIM-DMARC path shows that they are unwilling to bite the bullet of the obvious (and obviously correct) solution: proper per-author digital signatures by default.
DMARC, as far as I can see (and have previously argued), is a good optimization for corporate authors, where users of a mailbox in a domain are delegates of the corporate owner of the domain. Where that is not true, it sucks for a whole slew of reasons, yet AOL and Yahoo! are trying to apply it.
I wouldn't waste time worrying about whether various hacks might make it 0.0001% easier to phish people.
Will you please stop focusing on *your* logic, and start thinking about what happens if people with different interpretations of the facts take action on those interpretations?
My apologies. My imagination is sadly limited by 20 years of running mailing lists for real people, and extensive conversations with the people who designed and use DMARC.
Regards, John Levine, johnl@taugh.com, Taughannock Networks, Trumansburg NY Please consider the environment before reading this e-mail.
John R Levine writes:
My apologies. My imagination is sadly limited by 20 years of running mailing lists for real people, and extensive conversations with the people who designed and use DMARC.
Experience doesn't limit imagination, it's desperation to solve a difficult problem in a hurry that does that. That said, your years of experience are undoubtedly valuable, as is the information from your experiments with .INVALID.
I do not say you "should" take a more global view (I said "please", didn't I?) I'd like the benefit of your thoughts from that global point of view, but it's up to you whether you want to go to that effort.
I'm not trying to criticize you (though I've expressed myself badly -- I'm frustrated with the "damn the torpedos" atmosphere created by Yahoo! and AOL, that's also coloring a lot of thinking here). It's not your responsibility to think globally -- you're an operator with a problem to address, not responsible for creating a standard or for maintaining infrastructure that implements that standard. I support your experiments with .INVALID, although I believe them to be technically not conforming to RFC. "Code is law", but *RFCs aren't*.
What'm trying to do is explain why Mailman should (IMHO) take a quite different, much more conservative, stance toward implementing this, and why I criticize DMARC. The DMARC folks *are* creating a standard, and Mailman *is* attempting to implement it. It *is* *our* (and *their*!) responsibility to think globally, even as we act locally.
BTW, we've had conversations with several of those "designers and users" here, and they clearly do suffer from the "we have a problem we need to fix, and we'll deal with the consequences for other people -- which we don't think are a big deal -- later if need be" attitude. That includes several folks who do know better to do than what the big portals are doing, so I figure they're REALLY desperate (or at least the folks at email providers which are abusing p=reject are).
Note that AOL and Yahoo! need to do this because they have ambitions of being e-commerce platforms, and so their domain names can be used to scam money out of people. I don't think their business model justifies the harm their refusal to conform to RFC does to our subscribers.
What'm trying to do is explain why Mailman should (IMHO) take a quite different, much more conservative, stance toward implementing this, and why I criticize DMARC.
I don't know anyone who thinks the way that Yahoo and AOL are using DMARC is a good idea.
But you're nuts if you think that every Mailman list is going to kick off every Yahoo and AOL user, no matter how much the two providers might deserve it. As I said, some of us run lists to get work done rather than to make a political point, and the only way to continue to get work done at this point is to route around the DMARC damage somehow.
Note that AOL and Yahoo! need to do this because they have ambitions of being e-commerce platforms, and so their domain names can be used to scam money out of people.
We're deep enough into tin-foil hat territory here that we're done. Should you want to know what AOL and Yahoo are actually doing, see the most recent entry in my blog at http://jl.ly/
Regards, John Levine, johnl@taugh.com, Taughannock Networks, Trumansburg NY Please consider the environment before reading this e-mail.
John R Levine writes:
But you're nuts if you think that every Mailman list is going to kick off every Yahoo and AOL user,
You can stop the ad hominem innuendo right there (that's an RFC 2119 MUST NOT). There is plenty of documentary evidence on Mailman lists that I'm fully aware that that is neither possible nor desirable.
If you would like to address what *Mailman* should do in polite terms, I'm ready.
John R Levine writes:
Note that AOL and Yahoo! need to do this because they have ambitions of being e-commerce platforms, and so their domain names can be used to scam money out of people.
We're deep enough into tin-foil hat territory here that we're done. Should you want to know what AOL and Yahoo are actually doing, see the most recent entry in my blog at http://jl.ly/
*shrug* I'm a business school professor, and do analysis like the analysis behind my statement above for a living. It's a fairly obvious extension of my posts on use of DMARC by financial institutions. And it helps explain why GMail seems to be actively dissenting, and Hotmail has not followed yet. Those email providers have a different approach to being portals, and use different trade names for their financial activites to the extent that I am aware of them. I'm not going to yield on it quite yet.
So, call it "tin-foil hat territory" if you like. Being an expert on Internet business models doesn't make me *right*, but name-calling isn't going to bother me.
On May 04, 2014, at 01:26 PM, John Levine wrote:
I realize I'm a bit late to this party, but this is a technique that I don't think has been addressed here. On my lists I've fixed the DMARC bounces by rewriting From: lines of DMARC'ed domains like this on the way out:
From: Marissa <mmeyer@yahoo.com>
to
From: Marissa <mmeyer@yahoo.com.invalid>
I have some sympathy for this approach, as I mentioned over in mailman-users. It violates RFCs so I'm not sure Mailman should adopt it, but it's worth experimenting with, and I'm glad you (John) are doing so, and providing feedback here.
I'm not personally concerned about the effects of .invalid on phishing, since I largely agree with John's later statement that there are plenty of "pretty close" domains you can stick in the From header that will fool most non-technical users. Heck, I see dozens per day and some are clever enough to even fool me before close inspection reveals the subterfuge. Add to that, as others have observed, that many MUAs don't even display the actual email address.
Of course, adding .invalid doesn't really solve the problem, and I'm quite uncomfortable with overloading even more operations onto Reply-To. As seen on mailman-users, the interactions with the various options is a mess, difficult to get right, fragile, and difficult to understand all the implications.
Message wrapping is the safest but equally unsatisfying. It's pretty clear to me that there are *no* good solution today to DMARC's affect on mailing lists, only less bad ones.
-Barry
From: Marissa <mmeyer@yahoo.com.invalid>
I have some sympathy for this approach, as I mentioned over in mailman-users. It violates RFCs so I'm not sure Mailman should adopt it, but it's worth experimenting with, and I'm glad you (John) are doing so, and providing feedback here.
I know the guy who wrote the RFC, and I'm reasonably sure he'd forgive us under the circumstances. There is, after all, plenty of mail from addresses like donotreply@bigbank.com already, and this does the same thing with slightly different and syntax. Note that putting the list's address in From: is also an RFC violation since the list isn't the author by any normal understanding of the term.
Message wrapping is the safest but equally unsatisfying. It's pretty clear to me that there are *no* good solution today to DMARC's affect on mailing lists, only less bad ones.
The good solution is for the members of the DMARC group to set up and use a shared DMARC-bypass whitelist for mailing lists and other legitimate mail sources that don't match DMARC's narrow security model. I'm working on it, but in the meantime, .INVALID has the advantage of reminding people where the source of the problem lies.
R's, John
participants (7)
-
Barry Warsaw -
John Levine -
John R Levine
-
Mark Rousell -
Mark Sapiro -
SM -
Stephen J. Turnbull