Re: [Mailman-Developers] feature request: one-click setting to preserve DKIM
On Dec 06, 2011, at 10:17 AM, Murray S. Kucherawy wrote:
I think this is the one big lesson from these discussions: DKIM is mostly incompatible with mailing lists. Where the two must be integrated, some usability will likely be compromised.
It depends on your expectations. If there's an expectation that the author's signature will/should/must persist through a mailing list, then I agree that they're largely incompatible. If on the other hand you intend for lists to re-sign mail and for receivers to evaluate the message based on the list signature rather than the author signature, then it's entirely workable. Of course, sometimes the author signature will indeed survive, and then you have two domains to evaluate instead of one. Bonus!
My own personal feeling is that having lists re-sign messages is the best expectation to put forward. You're subscribed to a mailing list, so you trust that list much more than you trust the senders on that list. So having the mailing list site re-sign the outgoing messages seems to me to be best practice. My inclination is that removing the original author's signature first is not entirely inappropriate.
Note too that Mailman supports anonymizing list traffic to the extent that it would wipe out the original From header. Some lists turn this on for a higher degree of privacy than you see on most open discussion lists. In that case, the From header would look like it's coming from the mailing list, and then it would make the most sense to remove any original signature and leave only the list's signature.
It seems to me that relying on From header signing of mailing list messages just isn't that useful.
As Monica points out, DKIM itself is oblivious to the context in which it's being used.
I'm no DKIM expert by far, but it seems to me that a mailing list message has enough clues that a DKIM verifier could do a better job at helping recipients make good choices. For example, all Mailman messages have a List-Id header that contains the domain name hosting the list server. With re-signing, why couldn't you verify the signature against the List-ID host instead of the original sender?
You could. The issue isn't that doing so is wrong or impossible. It's simply that those semantics aren't built into DKIM, largely because we have no evidence that that's a useful test.
The ESP community has argued that third-party signatures (those different than the one in the From: field) shouldn't be valued any less than one that does match, for various reasons. That argument was apparently persuasive.
ADSP, and a draft I'm advancing called ATPS, extend DKIM's semantics to do more than just say "domain X handled this message" (which is really all a valid DKIM signature tells you). They both attempt to say things based on whether the domain delivered as part of a valid DKIM signature matches some identifier in the message, like the domain in the From: or in the List-ID field. If Mailman (or an MUA, or a filter, or something) implements such checks and finds that it's operationally beneficial to end users do so, then it could publish that mechanism in an RFC or elsewhere, and people could start to use it. The resistance isn't that this is a bad idea, but just that there's no evidence to back it up that would justify its standardization.
In that case, it would be interesting and feasible for you to perhaps line up some mailing list sites to enable this and gather some statistics. It would be pretty easy to do (I think) in Mailman 3 (though that's unreleased) and probably not hard to do in Mailman 2.
The trick, of course, is not just to do something like this, but to get MUA buy-in. That is, when a signature validates and it presents a domain name that matches some identifier, change the presentation of the message to show this in some meaningful way. And then make sure in doing so that you don't inadvertently discredit legitimate messages for which that's not true.
Right. So, Gmail is probably the 800lb MUA gorilla here. Monica, do you have any thoughts on how you could run such an experiment and find out what is most useful to your users?
Cheers, -Barry
My own personal feeling is that having lists re-sign messages is the best expectation to put forward. You're subscribed to a mailing list, so you trust that list much more than you trust the senders on that list. So having the mailing list site re-sign the outgoing messages seems to me to be best practice. My inclination is that removing the original author's signature first is not entirely inappropriate.
This is why Google Groups removes incoming DKIM signatures and re-signs, because chances that the original signature survives are vanishingly small given most people's list settings.
Note too that Mailman supports anonymizing list traffic to the extent that it would wipe out the original From header. Some lists turn this on for a higher degree of privacy than you see on most open discussion lists. In that case, the From header would look like it's coming from the mailing list, and then it would make the most sense to remove any original signature and leave only the list's signature.
If From is wiped out, great! Problem solved, at least for me.
The trick, of course, is not just to do something like this, but to get MUA buy-in. That is, when a signature validates and it presents a domain name that matches some identifier, change the presentation of the message to show this in some meaningful way. And then make sure in doing so that you don't inadvertently discredit legitimate messages for which that's not true.
Right. So, Gmail is probably the 800lb MUA gorilla here. Monica, do you have any thoughts on how you could run such an experiment and find out what is most useful to your users?
In a sense we are already experimenting here. For example, this year there are new UI warnings when the payload From says gmail, but the message is not signed by Gmail (https://mail.google.com/support/bin/answer.py?answer=185812).[1] This either appears as a "this message was sent via <DKIM or SPF domain>" informational bar or more serious warning, "this message may not have been sent by foo@gmail.com", if the message doesn't authenticate at all. Needless to say this is affecting lots of list traffic, and many people don't like it:
http://snowulf.com/2011/06/29/gmail-thinks-this-message-may-not-have-been-se... http://www.yellowjug.com/how-to/gmail-phishing-alert-mailman-mailing-lists-s... http://www.drake.org.uk/2011/06/googles-new-gmail-phishing-detection-system-...
The pipe-dream fix for this, at least as far as mailing lists go, is to do better mailing list detection on the recipient side and maintain a list of lists that the user belongs to for suppressing warnings. We can't just ignore all mail that has a List-Id, though, because that's much too easy to forge.
Thanks, Monica
[1] Why are we doing this? Well, it turns out that account hijacking has been a huge problem in the last couple of years, and along with theft of contact information phishing scams have gotten more sophisticated, appearing to come from people that you know. Since Gmail signs all outbound mail adding warnings was one easy way to warn users when they get mail from someone pretending to be their contact but not actually coming from Gmail.
Barry Warsaw writes:
My own personal feeling is that having lists re-sign messages is the best expectation to put forward. You're subscribed to a mailing list, so you trust that list much more than you trust the senders on that list.
But as Monica points out, sometimes you need to evaluate whether you trust the sender, because otherwise you need to trust *all* of the list's competence to evaluate senders, congruence of the list's trust policy with your own, *and* the honesty of the list's host adminstrators.
So having the mailing list site re-sign the outgoing messages seems to me to be best practice. My inclination is that removing the original author's signature first is not entirely inappropriate.
But that doesn't work in the case in point, unless you also change the from field to reflect the list's domain.
What do these DKIM-strict domains do with digests? Do they actually check the content (ie, individual messages) for source domain and verify their DKIM signatures?
If not, just have those people who aren't getting messages turn on digest mode with maximum frequency. :-)
Of course, all the phishers out there are reading this message, and will shortly be using this technique to phish gmail users, so you'll have to extend DKIM checks to the content of digests and forwards....
What really ought to be done is to format secured messages as multipart, and sign the overall header "From" and individual parts (perhaps identified by some kind of content ID). Then have the *MUA* (not the MTA!) check for alleged sender, and for highly-phishable alleged senders display *only* authenticated portions (plus maybe buttons to see unauthenticated content at user option).
Dream on, Steve ...
On Tue, Dec 6, 2011 at 8:45 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
What do these DKIM-strict domains do with digests? Do they actually check the content (ie, individual messages) for source domain and verify their DKIM signatures?
Typically the digest appears to come from the list, so that's ok. There's no way to verify the contents with DKIM anyway at that point, anyway.
If not, just have those people who aren't getting messages turn on digest mode with maximum frequency. :-)
:) I'm not too worried about digests. They tend to look pretty different from the average phish, even when they only contain one message.
Of course, all the phishers out there are reading this message, and will shortly be using this technique to phish gmail users, so you'll have to extend DKIM checks to the content of digests and forwards....
What really ought to be done is to format secured messages as multipart, and sign the overall header "From" and individual parts (perhaps identified by some kind of content ID). Then have the *MUA* (not the MTA!) check for alleged sender, and for highly-phishable alleged senders display *only* authenticated portions (plus maybe buttons to see unauthenticated content at user option).
Yeah, unfortunately pushing this problem to the MUA introduces nearly as many problems as it solves. At the MTA we can't really know what the MUA is going to display (even in Gmail's case, because some people fetch their mail and view with another client) so the only safe thing to do is to make sure that all of it verifies.
Thanks, Monica
When a message comes from a Mailman mailing list, the headers carry far more useful identifying information than the From: header. In fact, the purported sender isn't in the From: header, it's in the List-ID: header. It would be better if mail clients (like Gmail's web client) would display this more prominently than the From header. Also, it would be good if lists DKIM signed messages with d= value matching the List-ID header.
Then, the Gmail interface could indicate that the List-ID header was verified, but the From header wasn't.
Now, if Gmail also prominently exposed the list-unsubscribe header, then the EU regulations on marketing messages would be satisfied. In my view, hiding the unsubscribe information in a menu isn't enough to satisfy the "easy to use" requirement, though it seems that Gmail does better than most vendors in this respect.
Finally, if Mailman allowed users to choose whether to get the footer added, and subject munged, then Gmail users might avoid these issues anyway. Mailman might provide a list of domains for which this was the default behaviour, and site admins should be able to manage such a list. Mailman might even provide updates for this list. Of course, it's complicated: for example I use Gmail, but with a vanity domain. And then I use the IMAP interface with a client that doesn't expose list headers.
My view is that a one-click setting to preserve DKIM might be useful, but it should carry a health warning saying something like: "If you're an organisation in the EU, and this list helps to promote your organisation, or keep people in touch with your organisation, then selecting this option may be in breach of your country's mail privacy laws." In fact, it may be illegal if you simply have a list subscriber in the EU.
-- Ian Eiloart Postmaster, University of Sussex +44 (0) 1273 87-3148
On Jan 04, 2012, at 01:23 PM, Ian Eiloart wrote:
Finally, if Mailman allowed users to choose whether to get the footer added, and subject munged, then Gmail users might avoid these issues anyway.
Of course, such level of option would require personalization, i.e. one message per recipient. Maybe for most sites (but not all) the decade-old economics that pushed us toward avoiding personalization by default has changed enough now. MM3's architecture is more conducive to loading more functionality into personalized delivery.
What do you think about changing this default?
-Barry
On Thu, Jan 5, 2012 at 7:46 AM, Barry Warsaw <barry@list.org> wrote:
On Jan 04, 2012, at 01:23 PM, Ian Eiloart wrote:
Finally, if Mailman allowed users to choose whether to get the footer added, and subject munged, then Gmail users might avoid these issues anyway.
Of course, such level of option would require personalization, i.e. one message per recipient. Maybe for most sites (but not all) the decade-old economics that pushed us toward avoiding personalization by default has changed enough now. MM3's architecture is more conducive to loading more functionality into personalized delivery.
What do you think about changing this default?
Are you suggesting changing the default to not munging the subject and adding the footer? I would be in favor, but also be interested in surveying mailman users to see how many use clients that offer no support for identifying or unsubscribing from mailing list mail.
Thanks, Monica
Hi Ian,
On Wed, Jan 4, 2012 at 5:23 AM, Ian Eiloart <iane@sussex.ac.uk> wrote:
When a message comes from a Mailman mailing list, the headers carry far more useful identifying information than the From: header. In fact, the purported sender isn't in the From: header, it's in the List-ID: header. It would be better if mail clients (like Gmail's web client) would display this more prominently than the From header. Also, it would be good if lists DKIM signed messages with d= value matching the List-ID header.
Then, the Gmail interface could indicate that the List-ID header was verified, but the From header wasn't.
I agree that mailing list management needs a serious overhaul in the UI, for Gmail and for other clients.
Now, if Gmail also prominently exposed the list-unsubscribe header, then the EU regulations on marketing messages would be satisfied. In my view, hiding the unsubscribe information in a menu isn't enough to satisfy the "easy to use" requirement, though it seems that Gmail does better than most vendors in this respect.
The exposure is more than in the menu dropdown. If the message comes from a mailing list with good reputation, and the user clicks "Report spam", then Gmail offers to send an unsubscribe message to the mailto address in List-Unsubscribe. The reason we have the reputation check is that we do not want spammy senders to use the unsubscribe emails for list washing.
http://gmailblog.blogspot.com/2009/07/unsubscribing-made-easy.html
Finally, if Mailman allowed users to choose whether to get the footer added, and subject munged, then Gmail users might avoid these issues anyway. Mailman might provide a list of domains for which this was the default behaviour, and site admins should be able to manage such a list. Mailman might even provide updates for this list. Of course, it's complicated: for example I use Gmail, but with a vanity domain. And then I use the IMAP interface with a client that doesn't expose list headers.
My view is that a one-click setting to preserve DKIM might be useful, but it should carry a health warning saying something like: "If you're an organisation in the EU, and this list helps to promote your organisation, or keep people in touch with your organisation, then selecting this option may be in breach of your country's mail privacy laws." In fact, it may be illegal if you simply have a list subscriber in the EU.
Hmm, this seems difficult to enforce. How would I know if a list subscriber were in the EU? Even if the member address were obviously not hosted in the EU, it could easily forward to an EU address.
Thanks, Monica
participants (4)
-
Barry Warsaw -
Ian Eiloart -
Monica Chew -
Stephen J. Turnbull