Who authored the message?
Since DEMARC we don't know who is authoring list messages anymore.
I saw a message today sent to a list named "AlmostEverybody". The message author did not properly configure their MUA (Verizon webmail) with their proper name, so the only identifier is their email address. Which is of course now deleted as part of DEMARC compliance. Leaving us with a message and no indication of who sent it. The "From" was merely "Via AlmostEverybody@list.example.com". No name, no author email address. Of course no sig either.
List members are unable to reply off-list to the author, and they don't even know who the author is.
Would it be a reasonable feature request to add the author's name & email address as a X-Header? Some of us not only read the headers on a regular basis, but we even configure our MUA to display certain message Headers. (My favorites are Reply-To, X-Mailer, and User-Agent).
I'd rather not add the author's email address to the top or bottom of the message body, but it seems that some method of identifying the message author is in order. Even if depreciated, X-Headers are obviously still in use, and better then nothing. At the very least adding one more message header won't cause any complaining.
Thoughts?
Best, Dave Nathanson Mac Medix
On 05/12/2014 08:45 PM, Dave Nathanson wrote:
I saw a message today sent to a list named "AlmostEverybody". The message author did not properly configure their MUA (Verizon webmail) with their proper name, so the only identifier is their email address. Which is of course now deleted as part of DEMARC compliance. Leaving us with a message and no indication of who sent it. The "From" was merely "Via AlmostEverybody@list.example.com". No name, no author email address. Of course no sig either.
What Mailman version are you using? In the current version (2.1.18-1) you should be seeing the either the author's display name from her From: header or if none and From: a list member, the members real name from the membership list, or if none, at least the local part of the email address.
Also, the author's original From: will be in Reply-To: in every case, except see bug <https://bugs.launchpad.net/mailman/+bug/1318025>.
List members are unable to reply off-list to the author, and they don't even know who the author is.
Again, what is your Mailman version and what are your Reply-To munging settings. In versions older than 2.1.18, I think you should still see the author's address in Reply-To: if first_strip_reply_to is No.
Would it be a reasonable feature request to add the author's name & email address as a X-Header? Some of us not only read the headers on a regular basis, but we even configure our MUA to display certain message Headers. (My favorites are Reply-To, X-Mailer, and User-Agent).
I don't think it's necessary. If the author's address isn't in Reply-To:, it should be, and the absence is a bug. Is that not sufficient?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi Mark, We've got Mailman 2.1.17 at Dreamhost.
List Settings: from_is_list = Mung From anonymous_list = No first_strip_reply_to = Yes reply_goes_to_list = Explicit address reply_to_address = AlmostEverybody@example.com (without subdomain) Include_sender_header = Yes
In this case, the message author MUA does not provide their display name, and the list _does_ have a real name for that person, but Mailman did not insert it. So after going through Mailman, the From header said only: "Via AlmostEverybody@list.example.com".
Today I experimented with first_strip_reply_to set to No.
This means that a reply is addressed to both the list and to previous author. If nodups = Yes. Then the previous author will get only 1 copy of the message. But the copy they get is the direct mail (including senders email address), not the list mail. When that person replies to the message, it will only go to the most recent previous author, not to the list.
Maybe this is acceptable(?), since most of the time people don't reply to their own list messages?
Im my experience, the majority of users are incapable of adding or deleting anything from the Reply-To or To field. That's why up to now, I've opted for the Reply-To to only contain the desired list address.
Best, Dave Nathanson Mac Medix
On May 12, 2014, at 9:32 PM, Mark Sapiro <mark@msapiro.net> wrote:
On 05/13/2014 09:48 AM, Dave Nathanson wrote:
That's fixed in 2.1.18.
DMARC has forced mitigation responses. As far as I can tell, there are no ways to deal with this that don't involve impacts on message readability, replies or both other than not accepting messages From: domains with DMARC p=reject.
Im my experience, the majority of users are incapable of adding or deleting anything from the Reply-To or To field. That's why up to now, I've opted for the Reply-To to only contain the desired list address.
And these are the same users who would never see an X-Mailman-* header.
Note: that there are reasons for wanting replies to go to the poster and the list. E.g., the poster may be receiving digests so including her in replies keeps her in the loop without her having to wait for the next digest.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 05/13/2014 10:47 AM, Mark Sapiro wrote:
I have chosen, at least temporarily, to rewrite the from header as follows:
s/^(From:.*)([^ \t<>]+)@((yahoo|aol)\.com)/\1\2-AT-\3@mydomain.com/
So, addresses get rewritten as:
From: yahoousername-AT-yahoo.com@mydomain.com
and I do this only for domains which use p=reject and I make sure that there is always a reply-to header, since the From is no longer a valid email address.
My sense is that someone could come up with arguments as to why this is a bad idea, but so far I like what it looks like to the user better than other options I have seen.
I have not yet installed the 2.1.18 release though I hope to do that soon.
Nataraj
So, addresses get rewritten as:
From: yahoousername-AT-yahoo.com@mydomain.com
My sense is that someone could come up with arguments as to why this is a bad idea, ...
It's a bad idea for the same reason that all of the other anti-DMARC hacks are a bad idea, they break the existing usage of mail.
Under the current unpleasant circumstances, it's not much worse than any other, give or take what you do with the replies. Do you forward them back to the original user? Reject with a mysterious failure code? Discard them?
RFC nitpick: the mailbox part of an address is limited to 64 characters, so this has some risk of violating that limit, and there are a few MTAs that care. The domain part can be up to 256 which is why I put my noise there.
R's, John
On 05/19/2014 10:02 AM, John Levine wrote:
Thank you for your feedback. I'm most inclined to handle replies based on the needs of the particular list. Personally I find myself sending most replies to the list address and for small lists like the ones I run, I think that's the best choice. So I'm inclined to add/replace the reply to header to the list address. I know many high traffic lists prefer the reply to default directly to the sender. In that case, if there is an existing reply-to, I would keep that, otherwise, copy the original from header into the reply-to.
I run a mail client (thunderbird) which recognizes mailing lists, and so provides me with a reply and a reply-list button. My sense is that there are alot of mail clients that don't do that, so the default has to take that into consideration. I think the defaults should provide the best support for non-technical/inexperienced users. Yahoo, by default adds a reply-to header.
Ok, I will consider that, though if I really get mailbox names that long, maybe they should be treated as spam anyway.
R's, John
From: yahoousername-AT-yahoo.com@mydomain.com
No, I mean what will you do when people respond to your synthesized names? At some point you'll get mail at the server for mydomain.com for yahoousername-AT-yahoo.com@mydomain.com. What will you do with it?
One of the reasons I did the .invalid hack (which you can do with essentially the same code you're using) is that it's clear that the address isn't deliverable so there's no question of what happens to it.
R's, John
On 05/19/2014 04:49 PM, John Levine wrote:
I had previously missed the thread about the .INVALID thing (I'm not on the developers list). I have postfix configured so that the smtp server will return a 5XX response of "No such User". One difference between my method and yours is that my mail logs will show that somebody actually replied to that address where as with yours the reply would stop at the senders SMTP server. Not that significant, but it might be useful to know if users are using those addresses. I could add something that will make it clear that the addresses is not emailable, though my sense is that most users would get that the way it is, especially if they tried to email it and it failed.
Nataraj
Natu writes:
If you want that information you might as well run the forwarding service. You're imposing what may be significant costs for non- technical users, who are likely to be confused by the fact that your server at rjl.com is responding to a message they sent to aol.com.
But the percentage of AOL and Yahoo! users who would understand is likely to be much lower, if not a small minority. People use those services *because* they don't want to learn about email, they just want it to work. And the posters are likely to get upset if you make it hard for them to receive personal replies; a send -- DSN -- resend cycle may take a long time (especially if the sender has to ask technical support "WTF?" :-/ )
I thank my lucky star that almost none of my users use those domains, and so far 100% of those that do have thanked me for explaining the problem and switched to posting from GMail or whatever.
Steve
On Mon, 2014-05-12 at 20:45 -0700, Dave Nathanson wrote:
Since DEMARC we don't know who is authoring list messages anymore.
The use of DMARC p=reject by ESPs _implies_ either some loss of information, or the necessity to encapsulate a list post losslessly as a MIME spec'd attachment. The former solution unavoidably violates RFCs, while the latter is RFC compliant. But because the world is full of MUAs which don't handle the required Content-Type uniformly, and non-tech email users who are confused/put off by this, it may be judged to be a Bad Idea from a practical point of view.
As Mark pointed out, the best possible selection of current alternatives is available in MM 2.1.18-1, where both options are available.
My guess is that going forward, authenticated email of one sort or another will become more common, and that MIME encapsulation of contents, like double-boxing fragile items when shipping them, will be come a standard practice, requiring better standardization in MUAs as to how this is presented.
Email protocols were developed in an era of a kinder, gentler Internet where every SMTP server was an open relay and spamming and phishing were very much the exception rather than the rule. It's an incredible testament to the folks who designed these protocols that the Internet email system, arguably the most stressed of all Internet services, still works at all, but it's fairly obvious that something is going to have to change.
-- Lindsay Haisley | "Everything works if you let it" FMP Computer Services | 512-259-1190 | --- The Roadie http://www.fmp.com |
On 05/12/2014 08:45 PM, Dave Nathanson wrote:
I saw a message today sent to a list named "AlmostEverybody". The message author did not properly configure their MUA (Verizon webmail) with their proper name, so the only identifier is their email address. Which is of course now deleted as part of DEMARC compliance. Leaving us with a message and no indication of who sent it. The "From" was merely "Via AlmostEverybody@list.example.com". No name, no author email address. Of course no sig either.
What Mailman version are you using? In the current version (2.1.18-1) you should be seeing the either the author's display name from her From: header or if none and From: a list member, the members real name from the membership list, or if none, at least the local part of the email address.
Also, the author's original From: will be in Reply-To: in every case, except see bug <https://bugs.launchpad.net/mailman/+bug/1318025>.
List members are unable to reply off-list to the author, and they don't even know who the author is.
Again, what is your Mailman version and what are your Reply-To munging settings. In versions older than 2.1.18, I think you should still see the author's address in Reply-To: if first_strip_reply_to is No.
Would it be a reasonable feature request to add the author's name & email address as a X-Header? Some of us not only read the headers on a regular basis, but we even configure our MUA to display certain message Headers. (My favorites are Reply-To, X-Mailer, and User-Agent).
I don't think it's necessary. If the author's address isn't in Reply-To:, it should be, and the absence is a bug. Is that not sufficient?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi Mark, We've got Mailman 2.1.17 at Dreamhost.
List Settings: from_is_list = Mung From anonymous_list = No first_strip_reply_to = Yes reply_goes_to_list = Explicit address reply_to_address = AlmostEverybody@example.com (without subdomain) Include_sender_header = Yes
In this case, the message author MUA does not provide their display name, and the list _does_ have a real name for that person, but Mailman did not insert it. So after going through Mailman, the From header said only: "Via AlmostEverybody@list.example.com".
Today I experimented with first_strip_reply_to set to No.
This means that a reply is addressed to both the list and to previous author. If nodups = Yes. Then the previous author will get only 1 copy of the message. But the copy they get is the direct mail (including senders email address), not the list mail. When that person replies to the message, it will only go to the most recent previous author, not to the list.
Maybe this is acceptable(?), since most of the time people don't reply to their own list messages?
Im my experience, the majority of users are incapable of adding or deleting anything from the Reply-To or To field. That's why up to now, I've opted for the Reply-To to only contain the desired list address.
Best, Dave Nathanson Mac Medix
On May 12, 2014, at 9:32 PM, Mark Sapiro <mark@msapiro.net> wrote:
On 05/13/2014 09:48 AM, Dave Nathanson wrote:
That's fixed in 2.1.18.
DMARC has forced mitigation responses. As far as I can tell, there are no ways to deal with this that don't involve impacts on message readability, replies or both other than not accepting messages From: domains with DMARC p=reject.
Im my experience, the majority of users are incapable of adding or deleting anything from the Reply-To or To field. That's why up to now, I've opted for the Reply-To to only contain the desired list address.
And these are the same users who would never see an X-Mailman-* header.
Note: that there are reasons for wanting replies to go to the poster and the list. E.g., the poster may be receiving digests so including her in replies keeps her in the loop without her having to wait for the next digest.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 05/13/2014 10:47 AM, Mark Sapiro wrote:
I have chosen, at least temporarily, to rewrite the from header as follows:
s/^(From:.*)([^ \t<>]+)@((yahoo|aol)\.com)/\1\2-AT-\3@mydomain.com/
So, addresses get rewritten as:
From: yahoousername-AT-yahoo.com@mydomain.com
and I do this only for domains which use p=reject and I make sure that there is always a reply-to header, since the From is no longer a valid email address.
My sense is that someone could come up with arguments as to why this is a bad idea, but so far I like what it looks like to the user better than other options I have seen.
I have not yet installed the 2.1.18 release though I hope to do that soon.
Nataraj
So, addresses get rewritten as:
From: yahoousername-AT-yahoo.com@mydomain.com
My sense is that someone could come up with arguments as to why this is a bad idea, ...
It's a bad idea for the same reason that all of the other anti-DMARC hacks are a bad idea, they break the existing usage of mail.
Under the current unpleasant circumstances, it's not much worse than any other, give or take what you do with the replies. Do you forward them back to the original user? Reject with a mysterious failure code? Discard them?
RFC nitpick: the mailbox part of an address is limited to 64 characters, so this has some risk of violating that limit, and there are a few MTAs that care. The domain part can be up to 256 which is why I put my noise there.
R's, John
On 05/19/2014 10:02 AM, John Levine wrote:
Thank you for your feedback. I'm most inclined to handle replies based on the needs of the particular list. Personally I find myself sending most replies to the list address and for small lists like the ones I run, I think that's the best choice. So I'm inclined to add/replace the reply to header to the list address. I know many high traffic lists prefer the reply to default directly to the sender. In that case, if there is an existing reply-to, I would keep that, otherwise, copy the original from header into the reply-to.
I run a mail client (thunderbird) which recognizes mailing lists, and so provides me with a reply and a reply-list button. My sense is that there are alot of mail clients that don't do that, so the default has to take that into consideration. I think the defaults should provide the best support for non-technical/inexperienced users. Yahoo, by default adds a reply-to header.
Ok, I will consider that, though if I really get mailbox names that long, maybe they should be treated as spam anyway.
R's, John
From: yahoousername-AT-yahoo.com@mydomain.com
No, I mean what will you do when people respond to your synthesized names? At some point you'll get mail at the server for mydomain.com for yahoousername-AT-yahoo.com@mydomain.com. What will you do with it?
One of the reasons I did the .invalid hack (which you can do with essentially the same code you're using) is that it's clear that the address isn't deliverable so there's no question of what happens to it.
R's, John
On 05/19/2014 04:49 PM, John Levine wrote:
I had previously missed the thread about the .INVALID thing (I'm not on the developers list). I have postfix configured so that the smtp server will return a 5XX response of "No such User". One difference between my method and yours is that my mail logs will show that somebody actually replied to that address where as with yours the reply would stop at the senders SMTP server. Not that significant, but it might be useful to know if users are using those addresses. I could add something that will make it clear that the addresses is not emailable, though my sense is that most users would get that the way it is, especially if they tried to email it and it failed.
Nataraj
Natu writes:
If you want that information you might as well run the forwarding service. You're imposing what may be significant costs for non- technical users, who are likely to be confused by the fact that your server at rjl.com is responding to a message they sent to aol.com.
But the percentage of AOL and Yahoo! users who would understand is likely to be much lower, if not a small minority. People use those services *because* they don't want to learn about email, they just want it to work. And the posters are likely to get upset if you make it hard for them to receive personal replies; a send -- DSN -- resend cycle may take a long time (especially if the sender has to ask technical support "WTF?" :-/ )
I thank my lucky star that almost none of my users use those domains, and so far 100% of those that do have thanked me for explaining the problem and switched to posting from GMail or whatever.
Steve
On Mon, 2014-05-12 at 20:45 -0700, Dave Nathanson wrote:
Since DEMARC we don't know who is authoring list messages anymore.
The use of DMARC p=reject by ESPs _implies_ either some loss of information, or the necessity to encapsulate a list post losslessly as a MIME spec'd attachment. The former solution unavoidably violates RFCs, while the latter is RFC compliant. But because the world is full of MUAs which don't handle the required Content-Type uniformly, and non-tech email users who are confused/put off by this, it may be judged to be a Bad Idea from a practical point of view.
As Mark pointed out, the best possible selection of current alternatives is available in MM 2.1.18-1, where both options are available.
My guess is that going forward, authenticated email of one sort or another will become more common, and that MIME encapsulation of contents, like double-boxing fragile items when shipping them, will be come a standard practice, requiring better standardization in MUAs as to how this is presented.
Email protocols were developed in an era of a kinder, gentler Internet where every SMTP server was an open relay and spamming and phishing were very much the exception rather than the rule. It's an incredible testament to the folks who designed these protocols that the Internet email system, arguably the most stressed of all Internet services, still works at all, but it's fairly obvious that something is going to have to change.
-- Lindsay Haisley | "Everything works if you let it" FMP Computer Services | 512-259-1190 | --- The Roadie http://www.fmp.com |
participants (7)
-
Dave Nathanson -
Dave Nathanson
-
John Levine -
Lindsay Haisley -
Mark Sapiro -
Natu -
Stephen J. Turnbull