Hi all,
I'm one of the authors of the DKIM protocol and it recently came to my attention that you've recently changed mailman to remove DK and DKIM signature headers when you remail the message. This is incorrect behavior:
in Section 4:
Signers SHOULD NOT remove any DKIM-Signature header fields from
messages they are
signing, even if they know that the signatures cannot be verified.
This actually applies to everybody. There are several reasons for this. First is that DKIM allows you to specify the length of a body so it is not the case a priori that mailman will destroy the signature. Second, other heuristics can be applied to make mailing list traversal even better such as using the z= tag to determine whether trivial subject modifications have been made. Third and probably most important is that removing the signature is actually harmful rather than helpful: a broken signature and a missing signature MUST be treated as equivalent to no signature at all (lest an attacker just add a fake DKIM-signature header to get preferential treatment), and as above the verifier loses the ability to recover the signature.
Just as an FYI, we have deployed DKIM across all of Cisco and our successful mailing list traversal rate is about 99% -- a large percentage of which are through mailman lists. By making this change, you've taken the verify rate from 99% to 0% in one swell foop. Not good.
Mike
Hi Michael,
Thanks for writing about this. I suspect many are under the impression that passing messages through mail lists tended to break DomainKeys and DKIM (I know I was one, at least back when I was experimenting a lot with it). In fact, it always seemed to break on my Mailman lists, leaving what seemed to be a misleading result. It seemed that mail lists would need to "re-sign" messages as the new sender for it to work right, and did that not require the old key to be removed? Maybe things have changed recently, or maybe at least I misunderstood some aspects of it.
If, in fact, there is a way for the two to play well together, I think that would be great! It would remove one of the big "downsides" of using DK/DKIM. Since Mailman is one of the (if not *the* most) popular mail list systems, perhaps you can help Mailman developers to make the two things better integrated.
I would think/hope there would be a way to increase the handling/interaction to allow for 100% reliable flow-through of keys. That would really help DKIM's cause, I think, since so many people use mailing lists. Is this not theoretically possible if both Mailman and DKIM obey an understood set of rules? I believe that a solid way to "fix" this is a lot better than a stochastic one.
As for body length, would that not have to be a "guess"? I mean, short messages would fail due to the footer, and long ones would not get full checking (after the set length), right? If so, that sounds like a less than optimal solution.
I, for one, would love to see the email "accountability" problem solved. With spam becoming the one thing that could ruin the Internet, we need all the help we can get. I used DKIM for a while on my own server, but because of Mailman gotchas, etc. (and maybe these were more a problem of perception on my part), it seemed too easy to break. Also, with anything short of near global acceptance, it really seems like a tough battle, especially when so many are adopting things like SPF, which I never liked at all. Certainly, resolving the mailing list issue would be a huge step in the right direction, so consider me an enthusiastic supporter of your efforts.
-JoeMichael Thomas wrote:
Hi all,
I'm one of the authors of the DKIM protocol and it recently came to my attention that you've recently changed mailman to remove DK and DKIM signature headers when you remail the message. This is incorrect behavior:
in Section 4:
Signers SHOULD NOT remove any DKIM-Signature header fields from messages they are
signing, even if they know that the signatures cannot be verified.This actually applies to everybody. There are several reasons for this. First is that DKIM allows you to specify the length of a body so it is not the case a priori that mailman will destroy the signature. Second, other heuristics can be applied to make mailing list traversal even better such as using the z= tag to determine whether trivial subject modifications have been made. Third and probably most important is that removing the signature is actually harmful rather than helpful: a broken signature and a missing signature MUST be treated as equivalent to no signature at all (lest an attacker just add a fake DKIM-signature header to get preferential treatment), and as above the verifier loses the ability to recover the signature.
Just as an FYI, we have deployed DKIM across all of Cisco and our successful mailing list traversal rate is about 99% -- a large percentage of which are through mailman lists. By making this change, you've taken the verify rate from 99% to 0% in one swell foop. Not good.
Mike
Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/joe%40skyrush.com
Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp
Joe Peterson wrote:
Hi Michael,
Thanks for writing about this. I suspect many are under the impression that passing messages through mail lists tended to break DomainKeys and DKIM (I know I was one, at least back when I was experimenting a lot with it). In fact, it always seemed to break on my Mailman lists, leaving what seemed to be a misleading result. It seemed that mail lists would need to "re-sign" messages as the new sender for it to work right, and did that not require the old key to be removed? Maybe things have changed recently, or maybe at least I misunderstood some aspects of it.
I'm not sure whether Murray's dkim milter allows you to sign with the l= option or not, but setting the body length allows text to be appended to the end of a message -- like a mailing list trailer. This combined with some heuristics with subject line modification gets you to nearly 100% success rates. The only other problem that I've had with mailman is that it sometimes reorders the dkim-signature header field itself which causes the signature to break. I've been trying to figure out where in the code it does that so that we could either fix it or compensate for it when I'm creating the signature itself.
As for resigning -- that would certainly be a Good Thing, though not a cure-all. It's always better to have more accountability rather than less, but a valid first party signature (ie, From) is best of all. But in its absence, it would be nice for a remailer to sign the message so that it's possible for the receiver to whitelist them as a legit third party signer.
It's not entirely clear to me whether implementing it in the MTA or mailman is better, though I suppose that there would be no harm having it in both places as an option. It would probably be a pretty modest matter to implement a signer -- dealing with keys is really the biggest PITA.
If, in fact, there is a way for the two to play well together, I think that would be great! It would remove one of the big "downsides" of using DK/DKIM. Since Mailman is one of the (if not *the* most) popular mail list systems, perhaps you can help Mailman developers to make the two things better integrated.
Yes, that would be most excellent -- I've been meaning to subscribe for a while about these issues.
I would think/hope there would be a way to increase the handling/interaction to allow for 100% reliable flow-through of keys. That would really help DKIM's cause, I think, since so many people use mailing lists. Is this not theoretically possible if both Mailman and DKIM obey an understood set of rules? I believe that a solid way to "fix" this is a lot better than a stochastic one.
I've for quite a while thought that part of an ultimate DKIM BCP would give some guidance on what a "well behaved mailing list" would be. It would certainly be good if mailman were an example of that because at least at Cisco it accounts for the bulk of external mailing list traffic we see.
As for body length, would that not have to be a "guess"? I mean, short messages would fail due to the footer, and long ones would not get full checking (after the set length), right? If so, that sounds like a less than optimal solution.
No, the body length just counts the number of canonical bytes in the body when it's computing the hash, so that if there are bytes that are appended, it doesn't affect the outcome. The lack of the l= tag implies that the entire body is hashed which is most likely what the sendmail dkim milter does (at least by default). The main issue is that there is a security/robustness tradeoff with the use of l=. That is, bad guys could append content too. On the other hand, *if* that comes to pass, receivers are completely at liberty to scan the covered and uncovered parts of the body differently, delete the appended text, etc, etc.
I, for one, would love to see the email "accountability" problem solved. With spam becoming the one thing that could ruin the Internet, we need all the help we can get. I used DKIM for a while on my own server, but because of Mailman gotchas, etc. (and maybe these were more a problem of perception on my part), it seemed too easy to break. Also, with anything short of near global acceptance, it really seems like a tough battle, especially when so many are adopting things like SPF, which I never liked at all. Certainly, resolving the mailing list issue would be a huge step in the right direction, so consider me an enthusiastic supporter of your efforts.
Thanks, I appreciate that :) The one thing to be said is that you don't actually need to global acceptance to do useful things with dkim (though I'd certainly be happy if there were). The reason that Cisco deployed DKIM internally is to counter spear-phishing attacks. That is, forged mail that is a targeted social engineering attack. For this, we just need to determine whether the mail is or isn't from Cisco's servers and warn people when the message doesn't verify correctly. To pull this off, we need to keep the false positive rate fairly low which means that the legions of geeks using mailing list should mostly work. Which is why I was so alarmed to find out the newest versions of mailman was now doing the wrong thing.
In any case, backing out this change would go a *long* way to helping DKIM.
Mike
Michael Thomas wrote:
I'm not sure whether Murray's dkim milter allows you to sign with the l= option or not, but setting the body length allows text to be appended to the end of a message -- like a mailing list trailer. This combined with some heuristics with subject line modification gets you to nearly 100% success rates.
Consider that while Mailman doesn't do all of these things to every message, it can do any of the following:
- Add text to the beginning of the message body (msg_header)
- Add text to the end of the message body (msg_footer)
- Remove text from the beginning of the message body (Approved: line)
- Add additional MIME parts to a multipart message (msg_header, msg_footer)
- Convert a single part message to multipart in order to add msg_header/msg_footer
- Remove parts from a multipart message (content filtering)
- Convert an HTML part to plain text (content filtering)
- Decode a base64 or quoted-printable encoded part and perhaps re-encode it with a different encoding.
- Change or delete various headers including Subject:, To:, From:
- Replace some MIME parts with URLs of where they were stored and flatten the entire message into a single plain text message (scrubber).
- Probably other things I'm overlooking.
The only other problem that I've had with mailman is that it sometimes reorders the dkim-signature header field itself which causes the signature to break. I've been trying to figure out where in the code it does that so that we could either fix it or compensate for it when I'm creating the signature itself.
Other than the code which removes these headers, Mailman doesn't touch them. If they get modified in Mailman processing, it would have to be in the Python email library in the process of parsing the raw message into an email message object and then ultimately converting the message object back into raw text to be sent, but I don't think the email library does anything to headers that aren't explicitly processed in some way.
-- Mark Sapiro <msapiro@value.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro wrote:
Michael Thomas wrote:
I'm not sure whether Murray's dkim milter allows you to sign with the l= option or not, but setting the body length allows text to be appended to the end of a message -- like a mailing list trailer. This combined with some heuristics with subject line modification gets you to nearly 100% success rates.
Consider that while Mailman doesn't do all of these things to every message, it can do any of the following:
- Add text to the beginning of the message body (msg_header)
- Add text to the end of the message body (msg_footer)
- Remove text from the beginning of the message body (Approved: line)
- Add additional MIME parts to a multipart message (msg_header, msg_footer)
- Convert a single part message to multipart in order to add msg_header/msg_footer
- Remove parts from a multipart message (content filtering)
- Convert an HTML part to plain text (content filtering)
- Decode a base64 or quoted-printable encoded part and perhaps re-encode it with a different encoding.
- Change or delete various headers including Subject:, To:, From:
- Replace some MIME parts with URLs of where they were stored and flatten the entire message into a single plain text message (scrubber).
- Probably other things I'm overlooking.
Yes, there's no question that mailman as well as lots of other software can destroy signatures. In practice as people seem to actually use them, it is more theoretical than real. We've been running DKIM signers/verifiers for going on a year now and the 99% I quoted is across a 25000 user population which probably uses mailing lists far more than most similarly sized companies.
[note some of the things you mention can be worked around too]
The only other problem that I've had with mailman is that it sometimes reorders the dkim-signature header field itself which causes the signature to break. I've been trying to figure out where in the code it does that so that we could either fix it or compensate for it when I'm creating the signature itself.
Other than the code which removes these headers, Mailman doesn't touch them. If they get modified in Mailman processing, it would have to be in the Python email library in the process of parsing the raw message into an email message object and then ultimately converting the message object back into raw text to be sent, but I don't think the email library does anything to headers that aren't explicitly processed in some way.
That's something to go on. I haven't been able to figure out what exactly triggers this behavior because it doesn't appear to be very consistent.
MikeMichael Thomas wrote:
Yes, there's no question that mailman as well as lots of other software can destroy signatures. In practice as people seem to actually use them, it is more theoretical than real. We've been running DKIM signers/verifiers for going on a year now and the 99% I quoted is across a 25000 user population which probably uses mailing lists far more than most similarly sized companies.
I'm sure your statistics are valid for your environment, but I'm not sure that they are universally applicable. Consider what I think is a fairly typical situation exemplified by mailman-users@python.org. I don't know what fraction of incoming posts to this list are multipart/alternative with text/plain and text/html alternative parts, but I see many just from people who Cc: me directly.
It would be a fairly simple matter to go through the .mbox archive for any list that has one and count the number of X-Content-Filtered-By: Mailman/MimeDel and compare that to the number of messages. in fact, I just did that for a cycling club discussion list I managed, and just over 20% of the messages had content removed. Since the most common result of this is to throw away a text/html part and collapse the message to a single part, I submit that this will break a significant number of signatures.
No if the only result of this were that the recipient's MTA/MUAs considered these messages to be unsigned, that would be OK, but my understanding is that in some cases at least, these messages are either discarded or flagged as having invalid signatures. Either of these alternatives is not good. The former discards wanted messages, and the latter trains recipients to ignore the fact that signatures are invalid.
That said, it would be a simple matter to make the removal of these signature headers a site option (or even a list option, but I think a site option is more appropriate).
It would be better still to be able to make Mailman play better with DKIM so that we wouldn't have to break or remove signatures.
I note that Joe is one of the people who first identified the need to remove these headers. Perhaps together, we can find a better way.
See <http://sourceforge.net/tracker/index.php?func=detail&aid=1287546&group_id=103&atid=300103> for some discussion.
-- Mark Sapiro <msapiro@value.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Yep, for a time I was doing some testing of DKIM on my server (using the sendmail milter). )I was using sendmail at the time, and I have since switched to Postfix.) I did stop using DKIM after a while, and one reason was the mailing list stumbling block.
Since passing messages through Mailman appeared to always render a signature invalid, it certainly seemed correct to remove the sig from the header, thus allowing the outgoing MTA the chance to re-sign the message as the mail list server (not as good as an original sig, but at least it would pass the DKIM test at the receiving end). I know that one is supposed to treat an invalid signature the same as no signature, but knowingly letting email go out (and it is Mailman that is sending it) with a bad signature certainly felt "irresponsible" and potentially misleading.
Now, if it is indeed possible to come up with a way to make Mailman successfully pass the original signature through validly, that would be a big win - a real challenge I suspect. But I guess the big question now is what to do in the short term. I'm happy to back out the change, but perhaps we should let a few others weigh in on this topic first.
-JoeMark Sapiro wrote:
I note that Joe is one of the people who first identified the need to remove these headers. Perhaps together, we can find a better way.
I have demime in front of most of my larger lists, and I can tell you from casual peeks at the incoming copy that I keep, there are far too many people who send html email. My lists nuke all the html, so I'd say probably 75% of the incoming messages are modified.
I also would concur that deleting the DKIM sig would be the proper thing to do, as an invalid sig is sure to count towards spamminess. Its a challenge at times to get these emails delivered (I've pretty much given up on AOL, they are in such a mess), so anything like this that will trigger more unsuccessful deliveries is a real problem.
Bob
---------- Original Message ----------- From: Mark Sapiro <msapiro@value.net> To: Michael Thomas <mat@cisco.com> Cc: mailman-developers@python.org Sent: Thu, 1 Feb 2007 15:06:25 -0800 Subject: Re: [Mailman-Developers] dkim-signature headers
Michael Thomas wrote:
Yes, there's no question that mailman as well as lots of other software can destroy signatures. In practice as people seem to actually use them, it is more theoretical than real. We've been running DKIM signers/verifiers for going on a year now and the 99% I quoted is across a 25000 user population which probably uses mailing lists far more than most similarly sized companies.
I'm sure your statistics are valid for your environment, but I'm not sure that they are universally applicable. Consider what I think is a fairly typical situation exemplified by mailman-users@python.org. I don't know what fraction of incoming posts to this list are multipart/alternative with text/plain and text/html alternative parts, but I see many just from people who Cc: me directly.
It would be a fairly simple matter to go through the .mbox archive for any list that has one and count the number of X-Content-Filtered- By: Mailman/MimeDel and compare that to the number of messages. in fact, I just did that for a cycling club discussion list I managed, and just over 20% of the messages had content removed. Since the most common result of this is to throw away a text/html part and collapse the message to a single part, I submit that this will break a significant number of signatures.
No if the only result of this were that the recipient's MTA/MUAs considered these messages to be unsigned, that would be OK, but my understanding is that in some cases at least, these messages are either discarded or flagged as having invalid signatures. Either of these alternatives is not good. The former discards wanted messages, and the latter trains recipients to ignore the fact that signatures are invalid.
That said, it would be a simple matter to make the removal of these signature headers a site option (or even a list option, but I think a site option is more appropriate).
It would be better still to be able to make Mailman play better with DKIM so that we wouldn't have to break or remove signatures.
I note that Joe is one of the people who first identified the need to remove these headers. Perhaps together, we can find a better way.
See
<http://sourceforge.net/tracker/index.php?func=detail&aid=1287546&group_id=103&atid=300103>
for some discussion.
-- Mark Sapiro <msapiro@value.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/bob%40nleaudio.com
Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp ------- End of Original Message -------
We've been running with dkim signatures over a large population for nearly a year and have had no indication whatsoever that broken signatures do anything of the sort. Leaving the signatures in allows smarter receivers to have a chance to verify them, not to mention that deleting them destroys the forensic value of the signature.
I'm not sure that you all appreciate that resigning a message is no panacea. It's helpful, but only to a degree: a third party signature from a remailer is not a substitute for a first party signature from the From domain. If you have just a valid third party signature, the signing domain would need to be _specifically_ whitelisted by the receiver to be considered an acceptable signature lest you be subject to attacks by rogue third party signers. So to the degree that first party signatures can be validated the better.
MikeBob Puff wrote:
I have demime in front of most of my larger lists, and I can tell you from casual peeks at the incoming copy that I keep, there are far too many people who send html email. My lists nuke all the html, so I'd say probably 75% of the incoming messages are modified.
I also would concur that deleting the DKIM sig would be the proper thing to do, as an invalid sig is sure to count towards spamminess. Its a challenge at times to get these emails delivered (I've pretty much given up on AOL, they are in such a mess), so anything like this that will trigger more unsuccessful deliveries is a real problem.
Bob
---------- Original Message ----------- From: Mark Sapiro <msapiro@value.net> To: Michael Thomas <mat@cisco.com> Cc: mailman-developers@python.org Sent: Thu, 1 Feb 2007 15:06:25 -0800 Subject: Re: [Mailman-Developers] dkim-signature headers
Michael Thomas wrote:
Yes, there's no question that mailman as well as lots of other software can destroy signatures. In practice as people seem to actually use them, it is more theoretical than real. We've been running DKIM signers/verifiers for going on a year now and the 99% I quoted is across a 25000 user population which probably uses mailing lists far more than most similarly sized companies.
I'm sure your statistics are valid for your environment, but I'm not sure that they are universally applicable. Consider what I think is a fairly typical situation exemplified by mailman-users@python.org. I don't know what fraction of incoming posts to this list are multipart/alternative with text/plain and text/html alternative parts, but I see many just from people who Cc: me directly.
It would be a fairly simple matter to go through the .mbox archive for any list that has one and count the number of X-Content-Filtered- By: Mailman/MimeDel and compare that to the number of messages. in fact, I just did that for a cycling club discussion list I managed, and just over 20% of the messages had content removed. Since the most common result of this is to throw away a text/html part and collapse the message to a single part, I submit that this will break a significant number of signatures.
No if the only result of this were that the recipient's MTA/MUAs considered these messages to be unsigned, that would be OK, but my understanding is that in some cases at least, these messages are either discarded or flagged as having invalid signatures. Either of these alternatives is not good. The former discards wanted messages, and the latter trains recipients to ignore the fact that signatures are invalid.
That said, it would be a simple matter to make the removal of these signature headers a site option (or even a list option, but I think a site option is more appropriate).
It would be better still to be able to make Mailman play better with DKIM so that we wouldn't have to break or remove signatures.
I note that Joe is one of the people who first identified the need to remove these headers. Perhaps together, we can find a better way.
See
<http://sourceforge.net/tracker/index.php?func=detail&aid=1287546&group_id=103&atid=300103>
for some discussion.
-- Mark Sapiro <msapiro@value.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives:
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe:
http://mail.python.org/mailman/options/mailman-developers/bob%40nleaudio.com
Security Policy:
http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp ------- End of Original Message -------
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Feb 1, 2007, at 9:03 PM, Michael Thomas wrote:
We've been running with dkim signatures over a large population for
nearly a year and have had no indication whatsoever that broken signatures do anything of the sort. Leaving the signatures in allows smarter receivers to
have a chance to verify them, not to mention that deleting them destroys the
forensic value of the signature.
Mailman could also copy the original DKIM headers to something like X- Original-DKIM-Signature. That would preserve it (albeit in a non- standard way) for the forensic value.
- -Barry
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (Darwin)
iEYEARECAAYFAkXCuzkACgkQ2YZpQepbvXGbZACfQ28xB5znVIWVcBTrjfHU2jbx Z18An22cSHA9s+Zh0/Fi8/2fV4BqtijH =w5cO -----END PGP SIGNATURE-----
On 2/1/07 5:46 PM, "Bob Puff" <bob@nleaudio.com> wrote:
I have demime in front of most of my larger lists, and I can tell you from casual peeks at the incoming copy that I keep, there are far too many people who send html email.
Anyone using Windows machine, or a Mac starting with Tiger who hasn't made a change to the defaults out of the box sends HTML mail. Before Tiger, Mac users were sending text/rich. And there is little incentive to change the defaults unless one is an Internet veteran.
Plus AOL.
There is some self-selection away from HTML in the mailing list population, since many folks using lists know what's wrong with HMTL mail, and others get chided for sending it.
--John
Mark Sapiro wrote:
Michael Thomas wrote:
Yes, there's no question that mailman as well as lots of other software can destroy signatures. In practice as people seem to actually use them, it is more theoretical than real. We've been running DKIM signers/verifiers for going on a year now and the 99% I quoted is across a 25000 user population which probably uses mailing lists far more than most similarly sized companies.
I'm sure your statistics are valid for your environment, but I'm not sure that they are universally applicable. Consider what I think is a fairly typical situation exemplified by mailman-users@python.org. I don't know what fraction of incoming posts to this list are multipart/alternative with text/plain and text/html alternative parts, but I see many just from people who Cc: me directly.
Just to be clear, there are hacks that we do with the length such that mailing lists that insert trailers into mime structured posts still verify. This is done by not signing the trailing -- and/or </body></html>. This works pretty well. Obviously any wholesale conversions like 8->7bit or other suchlike are going to be a problem, but at least from our vantagepoint of being a pretty big company with a lot of mailing list use, it seems to be pretty rare.
It would be a fairly simple matter to go through the .mbox archive for any list that has one and count the number of X-Content-Filtered-By: Mailman/MimeDel and compare that to the number of messages. in fact, I just did that for a cycling club discussion list I managed, and just over 20% of the messages had content removed. Since the most common result of this is to throw away a text/html part and collapse the message to a single part, I submit that this will break a significant number of signatures.
Yes, that certainly would.
No if the only result of this were that the recipient's MTA/MUAs considered these messages to be unsigned, that would be OK, but my understanding is that in some cases at least, these messages are either discarded or flagged as having invalid signatures. Either of these alternatives is not good. The former discards wanted messages, and the latter trains recipients to ignore the fact that signatures are invalid.
We haven't seen anybody doing that, and I'd certainly hope that people don't use DKIM as the _sole_ indication of whether the message should be rejected; the false positive rate would be far too high. In any case, the follow on work from DKIM is the sender signing practices (SSP) work which in a nutshell allows a signer to publish whether they sign all of their mail or not. From the SSP standpoint, it makes no difference at all whether you strip the broken signature or not as both count as "no valid signature". I get the sense that people here think no signature would be better than a broken one, but I doubt that's how things will play out, especially with SSP.
That said, it would be a simple matter to make the removal of these signature headers a site option (or even a list option, but I think a site option is more appropriate).
I could live with that, especially if the default is to leave the signatures alone.
It would be better still to be able to make Mailman play better with DKIM so that we wouldn't have to break or remove signatures.
Right... what would be good here I think is to define a "dkim friendly" profile for the signing domain, mailing list configuration, as well as the receiver and where this kind of profile is appropriate. It would be nice if the default thing that mailman did was dkim friendly (which it mostly was until this change).
MikeI note that Joe is one of the people who first identified the need to remove these headers. Perhaps together, we can find a better way.
See <http://sourceforge.net/tracker/index.php?func=detail&aid=1287546&group_id=103&atid=300103> for some discussion.
Michael Thomas writes:
Just to be clear, there are hacks that we do with the length such that mailing lists that insert trailers into mime structured posts still verify. This is done by not signing the trailing -- and/or </body></html>. This works pretty well. Obviously any wholesale conversions like 8->7bit or other suchlike are going to be a problem, but at least from our vantagepoint of being a pretty big company with a lot of mailing list use, it seems to be pretty rare.
You don't work much with Japanese-language messages, obviously. *Most* of my Japanese mail gets translated in one direction or the other, and multiple translations are reasonably frequent for mailing lists. I suspect that you'll find that in mostly non-ASCII environments DKIM will have a lot more trouble.
Internationalization is definitely related to header-munging in Mailman lists. What apparently happens with I18N is that adjacent MIME encoded words get coalesced, and in particular linear white space gets dropped. Obviously leading spaces may get translated to tabs, etc, when doing RFC 822 wrapping on the way out. Also, there is no canonical way to do MIME encoding; it is perfectly legal to simply MIME encode the whole header even though there's only one Chinese character in it, or just that single character.
I would guess that DKIM headers are often fairly long. IIRC, the email module does the canonical thing (ie, just dropping the CRLF pair and leaving other linear white space alone) for wrapped headers. This will *probably* give you the identical headers when they're regenerated, but that's not guaranteed (eg, if there was trailing whitespace on some physical line in the original). Anyway, that's where I'd look first, if I had time to look.
That said, it would be a simple matter to make the removal of these signature headers a site option (or even a list option, but I think a site option is more appropriate).
If, as I suspect, I18N makes a difference, I would *definitely* want a list option. Most of my lists are English-only, but a few are for other languages and one is multilingual (sometimes in a single post! :-)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Feb 1, 2007, at 2:17 PM, Michael Thomas wrote:
I've for quite a while thought that part of an ultimate DKIM BCP would give some guidance on what a "well behaved mailing list" would be. It would certainly be good if mailman were an example of that because at least at Cisco it accounts for the bulk of external mailing list
traffic we see.
I agree with both statements. Note that there are many email related
RFCs that are ambiguous in the mailing list use case. We make
choices based on our best interpretation but it's never fully
satisfactory. If there's a possibility to have DKIM specify what a
properly behaving mailing should do (with of course, consensus from
this community and other listserver vendors), then I'm all for it.
(at least by default). The main issue is that there is a security/ robustness tradeoff with the use of l=. That is, bad guys could append content
too. On the other hand, *if* that comes to pass, receivers are
completely at liberty to scan the covered and uncovered parts of the body
differently, delete the appended text, etc, etc.
Isn't it possible that from the point of view of the original sender,
the mailing list /is/ the bad guy?
(Note too that of course it's trivial to disable DKIM header
cleansing in Cisco's own copy of Mailman.)
- -Barry
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (Darwin)
iEYEARECAAYFAkXCuiIACgkQ2YZpQepbvXF5OwCcCe2sET+qPrlQBMhwL9Aty9CL 6GEAn17BAMu9UC4p+mmUmigliEVDitQE =0INK -----END PGP SIGNATURE-----
Barry Warsaw wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Feb 1, 2007, at 2:17 PM, Michael Thomas wrote:
I've for quite a while thought that part of an ultimate DKIM BCP would give some guidance on what a "well behaved mailing list" would be. It would certainly be good if mailman were an example of that because at least at Cisco it accounts for the bulk of external mailing list traffic we see.
I agree with both statements. Note that there are many email related RFCs that are ambiguous in the mailing list use case. We make choices based on our best interpretation but it's never fully satisfactory.
If there's a possibility to have DKIM specify what a properly behaving mailing should do (with of course, consensus from this community and other listserver vendors), then I'm all for it.(at least by default). The main issue is that there is a security/robustness tradeoff with the use of l=. That is, bad guys could append content too. On the other hand, *if* that comes to pass, receivers are completely at liberty to scan the covered and uncovered parts of the body differently, delete the appended text, etc, etc.
Isn't it possible that from the point of view of the original sender, the mailing list /is/ the bad guy?
Absolutely! Consider:
From: barry@python.org Sender: fooledyou@badguy.com DKIM-Signature: d=badguy.com [...]
So you'd have a perfectly legitimate signature that corresponded with the Sender: badguy.com. I know who barry@python.org is but haven't a clue about badguy.com (or better, something just plain obscure). Would I want to bring it to the attention of the receiver (say in the MUA) "barry" is more well known now? Certainly not, because you're not. The only time you might consider that ok is if and only if you had some warm fuzzies about the third party signing. Which scales about as well as whitelists scale in general, which is to say that I would expect that we'll need a *lot* more experience to see how this actually will play out.
Note that this is not to discourage resigning by lists: it's cheap enough these days that if they're there for a reasonable % of lists, it will at least sow the ground for making whitelisting-like approaches more friendly (either local, aggregated...).
(Note too that of course it's trivial to disable DKIM header cleansing in Cisco's own copy of Mailman.)
Right -- we found this on some non-Cisco run lists though.
Mike-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Feb 1, 2007, at 11:54 AM, Michael Thomas wrote:
in Section 4:
Signers SHOULD NOT remove any DKIM-Signature header fields from messages they are signing, even if they know that the signatures cannot be verified.
This actually applies to everybody. There are several reasons for
this. First is that DKIM allows you to specify the length of a body so it is
not the case a priori that mailman will destroy the signature. Second, other heuristics can be applied to make mailing list traversal even better such as using
the z= tag to determine whether trivial subject modifications have been made.
Third and probably most important is that removing the signature is actually harmful rather than helpful: a broken signature and a missing signature MUST be
treated as equivalent to no signature at all (lest an attacker just add a fake DKIM-signature header to get preferential treatment), and as above the verifier loses the ability to recover the signature.
I haven't had time to read the formal spec, but does it specifically
handle the mailing list use case, where you cannot vouch for what
comes out the other end when you send a message to it?
- -Barry
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (Darwin)
iEYEARECAAYFAkXCuBIACgkQ2YZpQepbvXFurQCgknmNA6pJC/RTdSCFNb3Eoygw u/EAoJYm/SEPQ+KG5NhACNWzt5zjbwp0 =ri/f -----END PGP SIGNATURE-----
participants (7)
-
Barry Warsaw -
Bob Puff -
Joe Peterson -
John W. Baxter -
Mark Sapiro -
Michael Thomas -
Stephen J. Turnbull