On 25/05/2014 18:27, Peter Shute wrote:

I'm not comfortable with an email address in the display name not matching the real address. If I saw that in a non list email, it would look spammy to me.

I agree with you in the case of a non-list email but for a list email it seems to me to make perfect sense to include the original author's email address in the comment section and the list's address in the address section.

I don't like the idea of users getting used to seeing that sort of thing as normal, and there's the problem that lots of mail clients will only show the first few characters of the long display name. Would this be better: From: " [some-mail-list] Mark Rousell markr@signal100.com <some-mail-list@yahoogroups.com>

If a mail client only shows the first few characters then I think putting the mailing list name first would not be beneficial since users would not be able to see who wrote the message. The original author's name really does need to go first in my opinion.

I was alerted to this issue by complaints by the users of a Yahoo Groups mailing list of which I am a member. They are using Thunderbird which by default does an address book lookup on the From address. Naturally this was showing the mailing list name as the message author (for those of them with the mailing list in their address books) and not the original author. When the setting ('Show only display name for people in my address book') was disabled, then they could see the raw contents of the

From header. Thus the original author's name really does need to appear first but it would be even better if the mail client recognised the X-Original-From header.

-- Mark Rousell

PGP public key: http://www.signal100.com/markr/pgp Key ID: C9C5C162

On 25/05/2014 18:48, Richard Damon wrote:

On 5/25/14, 11:48 AM, Mark Rousell wrote: My view is that any attempt to have the Mail User Agent show a message that went through a mailing list as if it originated from the original poster (and only from that poster) is doomed, because the whole point of DMARC, is that domains that that are using DMARC are indicating that email that appears to be from their domain is only supposed to get to you from that domain, and others aren't supposed to be able to masquerade as it.

Three initial responses to this occur to me:

(1) Who says that DMARC will be successful? (Note that I am not opposed to DMARC or its intent but there is no guarantee of its long term success).

(2) In any case, this is not the problem of mailing lists or mail clients.

(3) Both forms of munging (both Yahoo's and the .INVALID approach), with or without the X-Original-From header, do *not* make it seem as if a message did not go through a mailing list. Indeed, they are simply ways of making it clear who the original author of a message was whilst not hiding that the message went through a mailing list.

See also below: I *do not* in fact think that this attacks DMARC.

If mailing list could do it, so could phishers.

Scammers and spammers do what they can. They will do what they can regardless of what other people do in order to continue operating effectively.

One might also observe that if "p=reject" DMARC users effectively externalise some aspects of their spam problem, it seems only appropriate and pragmatic for the rest of us to similarly externalise the problems so created.

And if this became somewhat common, DMARC, to achieve its goal of confirming sender identity would need to add checking that.

Not necessarily. There are two aspects to DMARC's protection mechanism: (1) What users of email from DMARC policy publishing domains see, and (2) what mail servers do before the user sees the email (if the user sees it at all).

Whilst mail client recognition of the X-Original-From header would alter what users see (which is in fact a key goal in this context, not a bug), DMARC would nevertheless still be effective in terms of its own design goals in that mail servers could still adhere to DMARC and reject or spamfilter non-compliant messages.

Why would this be harmless for mail lists but bad for spam? Because From munging and mail client parsing of the X-Original-From header will not impact on the automated effectiveness of DMARC in preventing users from seeing spam: Spammers would still be left with the difficulty of obtaining a valid DKIM header that aligns with their chosen From address whereas legitimate mail lists would not have this problem.

To put it another way, spammers might well use X-Original-From but this would not help them to get end users to see their spam emails if mail servers block or spamfilter non-DMARC-compliant messages (whereas mail lists emails would get through fine).

The only possible option for that would be wrapping, where the wrapped message was EXACTLY as sent (thus no content filtering, and all additions are added to the wrapper, not the original message).

Yes, wrapping would be ideal but I reckon this would be even more difficult in terms of mail client acceptance than X-Original-From would be. In other words, getting mail clients to correctly view wrapped messages is not going to happen any time soon but recognising and parsing X-Original-From (if it is present) is an easier thing to code.

The problem we are seeing is that we have domains that don't really need that restriction are using it, and the world is needing to figure out how to make this work.

I agree and the above (a combination of semantically useful From munging and a programmatically parseable X-Original-From header that does *not* interfere with automated DMARC filtering by mail servers) seems to me to be the basis of an effective and de facto-standardisable workaround to it.

The real solution, in my opinion, would be a standard that deals with how MUA display the senders of messages, and a system similar to https certificates where domains (or email addresses) that really need verification could get a certificate and be able to have their addresses displayed a "verified", email pretending to be from them but not properly signed could be rejected or marked suspicious, but most personal mail would just be unverified. It should make it clear that any ESP that uses this has an obligation to let is users understand the rules, and that it users are not allowed to let 3rd parties (like mailing list or other 3rd party mailers) send on their behalf (unless the standard allows a given email address to provide a cert to a 3rd party it indicate they are an authorized remailer for them). This should deal with the phish problem, at least once people are taught to look for the verified icon from "important" sources.

A good idea in theory but I suspect it falls down on the user education part. Most users don't learn.

The number of parties who would need to convert to it is also problematic, but conceivably something like this might be a part of some entirely new mail system, an as yet imaginary "MTPng" or similar.

-- Mark Rousell

PGP public key: http://www.signal100.com/markr/pgp Key ID: C9C5C162

Until spammers figure out they can send mail

From: spammer@evildomain.com X-Original-From: whatever@yahoo.com

This is one of the most annoying things about Yahoo and AOL's misuse of DMARC -- they're practically forcing people to use hacks to show unauthenticated fake From: lines.

R's, John

John Levine writes:

This is one of the most annoying things about Yahoo and AOL's misuse of DMARC -- they're practically forcing people to use hacks to show unauthenticated fake From: lines.

Not only that, they're doing it themselves. :-(

On 05/25/2014 11:35 PM, Mark Rousell wrote:

On 26/05/2014 07:24, Stephen J. Turnbull wrote:

...

John Levine writes:

...

This is one of the most annoying things about Yahoo and AOL's misuse of DMARC -- they're practically forcing people to use hacks to show unauthenticated fake From: lines.

Not only that, they're doing it themselves. :-(

To be fair, they are only including X-Original-From in their mail lists. No one actually parses it yet as far as I know.

They (Yahoo) are also including the original From: display name and email address in the display name of their list's munged From:. Given that some (notably mainstream Microsoft) email clients only show the display name as From:, this seems to me to totally defeat the intent of DMARC.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

a) It seems to me that this or something like it (i.e. new de facto standard headers to work around the problem) is surely an almost inevitable outcome anyway.

I wouldn't count on it. The reasonable approach to this kind of nonsense is for the relatively small set of ISPs using DMARC policy to whitelist the mail they already know it mishandles.

b) The way things are going all domains will sooner or later publish a DMARC policy if they want their mail to be accepted anywhere.

No, not at all. Comcast sent out a press release specifically saying that they have no plans to publish a DMARC policy record. Remember that AOL and Yahoo had huge security breaches in which crooks stole customer info including address books, and they're misusing DMARC as as panic reaction to try to compensate, sort of.

Let's just say this move hasn't made any them friends in the industry.

R's, John

On 25/05/2014 18:27, Peter Shute wrote:

I'm not comfortable with an email address in the display name not matching the real address. If I saw that in a non list email, it would look spammy to me.

I agree with you in the case of a non-list email but for a list email it seems to me to make perfect sense to include the original author's email address in the comment section and the list's address in the address section.

I don't like the idea of users getting used to seeing that sort of thing as normal, and there's the problem that lots of mail clients will only show the first few characters of the long display name. Would this be better: From: " [some-mail-list] Mark Rousell markr@signal100.com <some-mail-list@yahoogroups.com>

If a mail client only shows the first few characters then I think putting the mailing list name first would not be beneficial since users would not be able to see who wrote the message. The original author's name really does need to go first in my opinion.

I was alerted to this issue by complaints by the users of a Yahoo Groups mailing list of which I am a member. They are using Thunderbird which by default does an address book lookup on the From address. Naturally this was showing the mailing list name as the message author (for those of them with the mailing list in their address books) and not the original author. When the setting ('Show only display name for people in my address book') was disabled, then they could see the raw contents of the

From header. Thus the original author's name really does need to appear first but it would be even better if the mail client recognised the X-Original-From header.

-- Mark Rousell

PGP public key: http://www.signal100.com/markr/pgp Key ID: C9C5C162

On 25/05/2014 18:48, Richard Damon wrote:

On 5/25/14, 11:48 AM, Mark Rousell wrote: My view is that any attempt to have the Mail User Agent show a message that went through a mailing list as if it originated from the original poster (and only from that poster) is doomed, because the whole point of DMARC, is that domains that that are using DMARC are indicating that email that appears to be from their domain is only supposed to get to you from that domain, and others aren't supposed to be able to masquerade as it.

Three initial responses to this occur to me:

(1) Who says that DMARC will be successful? (Note that I am not opposed to DMARC or its intent but there is no guarantee of its long term success).

(2) In any case, this is not the problem of mailing lists or mail clients.

(3) Both forms of munging (both Yahoo's and the .INVALID approach), with or without the X-Original-From header, do *not* make it seem as if a message did not go through a mailing list. Indeed, they are simply ways of making it clear who the original author of a message was whilst not hiding that the message went through a mailing list.

See also below: I *do not* in fact think that this attacks DMARC.

If mailing list could do it, so could phishers.

Scammers and spammers do what they can. They will do what they can regardless of what other people do in order to continue operating effectively.

One might also observe that if "p=reject" DMARC users effectively externalise some aspects of their spam problem, it seems only appropriate and pragmatic for the rest of us to similarly externalise the problems so created.

And if this became somewhat common, DMARC, to achieve its goal of confirming sender identity would need to add checking that.

Not necessarily. There are two aspects to DMARC's protection mechanism: (1) What users of email from DMARC policy publishing domains see, and (2) what mail servers do before the user sees the email (if the user sees it at all).

Whilst mail client recognition of the X-Original-From header would alter what users see (which is in fact a key goal in this context, not a bug), DMARC would nevertheless still be effective in terms of its own design goals in that mail servers could still adhere to DMARC and reject or spamfilter non-compliant messages.

Why would this be harmless for mail lists but bad for spam? Because From munging and mail client parsing of the X-Original-From header will not impact on the automated effectiveness of DMARC in preventing users from seeing spam: Spammers would still be left with the difficulty of obtaining a valid DKIM header that aligns with their chosen From address whereas legitimate mail lists would not have this problem.

To put it another way, spammers might well use X-Original-From but this would not help them to get end users to see their spam emails if mail servers block or spamfilter non-DMARC-compliant messages (whereas mail lists emails would get through fine).

The only possible option for that would be wrapping, where the wrapped message was EXACTLY as sent (thus no content filtering, and all additions are added to the wrapper, not the original message).

Yes, wrapping would be ideal but I reckon this would be even more difficult in terms of mail client acceptance than X-Original-From would be. In other words, getting mail clients to correctly view wrapped messages is not going to happen any time soon but recognising and parsing X-Original-From (if it is present) is an easier thing to code.

The problem we are seeing is that we have domains that don't really need that restriction are using it, and the world is needing to figure out how to make this work.

I agree and the above (a combination of semantically useful From munging and a programmatically parseable X-Original-From header that does *not* interfere with automated DMARC filtering by mail servers) seems to me to be the basis of an effective and de facto-standardisable workaround to it.

The real solution, in my opinion, would be a standard that deals with how MUA display the senders of messages, and a system similar to https certificates where domains (or email addresses) that really need verification could get a certificate and be able to have their addresses displayed a "verified", email pretending to be from them but not properly signed could be rejected or marked suspicious, but most personal mail would just be unverified. It should make it clear that any ESP that uses this has an obligation to let is users understand the rules, and that it users are not allowed to let 3rd parties (like mailing list or other 3rd party mailers) send on their behalf (unless the standard allows a given email address to provide a cert to a 3rd party it indicate they are an authorized remailer for them). This should deal with the phish problem, at least once people are taught to look for the verified icon from "important" sources.

A good idea in theory but I suspect it falls down on the user education part. Most users don't learn.

The number of parties who would need to convert to it is also problematic, but conceivably something like this might be a part of some entirely new mail system, an as yet imaginary "MTPng" or similar.

-- Mark Rousell

PGP public key: http://www.signal100.com/markr/pgp Key ID: C9C5C162

Until spammers figure out they can send mail

From: spammer@evildomain.com X-Original-From: whatever@yahoo.com

This is one of the most annoying things about Yahoo and AOL's misuse of DMARC -- they're practically forcing people to use hacks to show unauthenticated fake From: lines.

R's, John

John Levine writes:

This is one of the most annoying things about Yahoo and AOL's misuse of DMARC -- they're practically forcing people to use hacks to show unauthenticated fake From: lines.

Not only that, they're doing it themselves. :-(

On 05/25/2014 11:35 PM, Mark Rousell wrote:

On 26/05/2014 07:24, Stephen J. Turnbull wrote:

...

John Levine writes:

...

This is one of the most annoying things about Yahoo and AOL's misuse of DMARC -- they're practically forcing people to use hacks to show unauthenticated fake From: lines.

Not only that, they're doing it themselves. :-(

To be fair, they are only including X-Original-From in their mail lists. No one actually parses it yet as far as I know.

They (Yahoo) are also including the original From: display name and email address in the display name of their list's munged From:. Given that some (notably mainstream Microsoft) email clients only show the display name as From:, this seems to me to totally defeat the intent of DMARC.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

a) It seems to me that this or something like it (i.e. new de facto standard headers to work around the problem) is surely an almost inevitable outcome anyway.

I wouldn't count on it. The reasonable approach to this kind of nonsense is for the relatively small set of ISPs using DMARC policy to whitelist the mail they already know it mishandles.

b) The way things are going all domains will sooner or later publish a DMARC policy if they want their mail to be accepted anywhere.

No, not at all. Comcast sent out a press release specifically saying that they have no plans to publish a DMARC policy record. Remember that AOL and Yahoo had huge security breaches in which crooks stole customer info including address books, and they're misusing DMARC as as panic reaction to try to compensate, sort of.

Let's just say this move hasn't made any them friends in the industry.

R's, John