Re: [Mailman-Developers] MIME footers

Hi Murray--
On Fri 2015-02-27 14:46:40 -0500, Murray S. Kucherawy wrote:
:)
I'm not one to make any decisions for mailman, but this sounds pretty reasonable to me. In fact, the only argument against ever having such a switch *not* set is that non-MIME-aware text-only MUAs will now see some crufty MIME boundaries in the message that they didn't see before.
I'd bet the number of users of non-MIME-aware text-only MUAs receiving mail from mailman that are incapable of wading through a bit of extra MIME boundary info on otherwise text/plain messages is vanishingly small. Most of those folks are probably at least used to seeing it as a result of the overwhelming predominance of multipart/alternative on today's SMTP network.
right, the MIME boundaries provide the parsing framework to enable this kind of sub-part DKIM-verification.
One issue this process brings up is that it's now necessary to treat pretty much every part of the message as though it is multipart/signed -- that is, it needs to be handled bytewise opaquely, on pain of breaking the DKIM header.
We already know that there are tools that do things like re-encode messages from base64 to q-p, or change character sets, or just re-flow long lines (FSVO "long") in the mime boundary subheaders. They're not supposed to do this with multipart/signed messages, because the RFCs point out that opaque handling is required, but they sometimes break them anyway. I imagine the same risk would now apply to all DKIM-signed messages using your scheme.
With that caveat in mind, it's clear that without your proposal, all DKIM-signatures of messages with footers will break anyway. So this does seem to be a strict improvement, even if there are a few corner cases that aren't fully covered.
--dkg

Daniel Kahn Gillmor writes:
Not vanishing. I'll add my testimony to Mark's that we see complaints related to added MIME parts (mostly from people who read the text/html alternative when it is kept, I think, but there are some people still using broken old MUAs, and the real hurt is when somebody's boss's MUA doesn't display the legal BS at the bottom -- neither the boss nor his lawyers are interested in "but we can't do anything about broken MUAs").
For some reason, no, that doesn't seem to be the case.

OK, this time I was properly caffeinated, but that also meant I had a short attention span. :-) Sorry for the long delay replying.
On Fri, Feb 27, 2015 at 12:55 PM, Daniel Kahn Gillmor <dkg@fifthhorseman.net
wrote:
The proposal I cobbled together (which is now in an expired IETF draft; I need to post an update so it's visible again, late this month probably) actually required that the canonicalized form of a MIME part is the fully decoded form. That way, 7bit, 8bit, q-p and base64 re-encodings would have no effect on the produced hash, and the signature would survive. Theoretically.
The tricky part is going to be the question of how to sign something that's not MIME, and then how to handle stuff like MIME preambles and postambles. There's going to be a lot of fussing over what various MUAs do with such content, and whether that should be signed, and so on. The best I think we can do is sign the actual part content, and be exceptionally clear about what leading and trailing spaces get included or omitted, etc. One byte of ambiguity and you're toast. That said, DKIM itself did a pretty good job of this, so we have that to use as a model.
And, if I may reply to some other points made on this thread:
As to the question of crusty MUAs, I'm inclined to agree with the "vanishingly small" characterization, except that as is typical in these situations, it's a rather vocal small. I know for example that alpine shows the preamble even though one could argue that it's not a usable part of a MIME message. Certainly the goal is to be as incremental and non-disruptive as possible, so I'd be inclined to do what someone suggested, which is to make it a flag (a list flag, or maybe a user flag), defaulting to something sane. But at some point we have to give up here, I feel; stuff like this becomes an attack directly on the MUA, and I would argue that there's only so much an MLM can (or should) do to protect users from crappy software.
I had forgotten about message headers (i.e., prepended text). Are those common? I had thought pretty much everyone uses footers only. It's certainly the case that this proposal only deals well with footers. The specific algorithm is to construct a MIME tree and sign parts of it; specifically, sign all of it, and then verify all of what you get first. The changed part would be in a new leaf, which is easy to exclude from hash generation. Excluding a part at or near the top might not be so straightforward. Anyway, that's an answer to Mark's "how do you determine what got signed?" question.
To Mark's second question, the idea is that the final receiver would get two signatures: The first was added by the author and signed the author-generated content, and the second was added by the MLM and included the author content plus a footer in such a way that the two are separated in a structured way (MIME boundaries) in contrast to DKIM's simple (and deprecated) "l=" tag. The MLM signed the whole thing, while the author signed only the original content. We like to say that DKIM allows a domain to take some responsibility for a message and its content; you could then determine for which parts each of the two parties is responsible. Translate this into the DMARC world: I could now tell that this message was signed by the author, and this added part was signed by an MLM which probably means the MLM added the extra part, which contains only text, so now DKIM passes, and DMARC is satisfied.
A "real world" example: I'm A at a domain with a "reject" DMARC policy, mailman's running at B, and subscriber C runs at a domain that checks DMARC inbound. I send mail to a list at B, and sign it with my domain's signature; the MLM at B appends a footer in the MIME way and re-signs a message using this list-aware canonicalization; C confirms that (1) most of the content is signed by A, (2) one added text/plain or text/html part was added by B, (3) B doesn't have a scary reputation or isn't blacklisted, and (4) B's signature covered the added part; C considers that a DKIM pass for A, which is "aligned" (in DMARC terms), and thus DMARC is satisfied.
In theory, to deal with the non-MIME case, we could invoke DKIM's "l=" tag, but it would be an understatement to say the community would be pretty allergic to that idea.
-MSK

Daniel Kahn Gillmor writes:
Not vanishing. I'll add my testimony to Mark's that we see complaints related to added MIME parts (mostly from people who read the text/html alternative when it is kept, I think, but there are some people still using broken old MUAs, and the real hurt is when somebody's boss's MUA doesn't display the legal BS at the bottom -- neither the boss nor his lawyers are interested in "but we can't do anything about broken MUAs").
For some reason, no, that doesn't seem to be the case.

OK, this time I was properly caffeinated, but that also meant I had a short attention span. :-) Sorry for the long delay replying.
On Fri, Feb 27, 2015 at 12:55 PM, Daniel Kahn Gillmor <dkg@fifthhorseman.net
wrote:
The proposal I cobbled together (which is now in an expired IETF draft; I need to post an update so it's visible again, late this month probably) actually required that the canonicalized form of a MIME part is the fully decoded form. That way, 7bit, 8bit, q-p and base64 re-encodings would have no effect on the produced hash, and the signature would survive. Theoretically.
The tricky part is going to be the question of how to sign something that's not MIME, and then how to handle stuff like MIME preambles and postambles. There's going to be a lot of fussing over what various MUAs do with such content, and whether that should be signed, and so on. The best I think we can do is sign the actual part content, and be exceptionally clear about what leading and trailing spaces get included or omitted, etc. One byte of ambiguity and you're toast. That said, DKIM itself did a pretty good job of this, so we have that to use as a model.
And, if I may reply to some other points made on this thread:
As to the question of crusty MUAs, I'm inclined to agree with the "vanishingly small" characterization, except that as is typical in these situations, it's a rather vocal small. I know for example that alpine shows the preamble even though one could argue that it's not a usable part of a MIME message. Certainly the goal is to be as incremental and non-disruptive as possible, so I'd be inclined to do what someone suggested, which is to make it a flag (a list flag, or maybe a user flag), defaulting to something sane. But at some point we have to give up here, I feel; stuff like this becomes an attack directly on the MUA, and I would argue that there's only so much an MLM can (or should) do to protect users from crappy software.
I had forgotten about message headers (i.e., prepended text). Are those common? I had thought pretty much everyone uses footers only. It's certainly the case that this proposal only deals well with footers. The specific algorithm is to construct a MIME tree and sign parts of it; specifically, sign all of it, and then verify all of what you get first. The changed part would be in a new leaf, which is easy to exclude from hash generation. Excluding a part at or near the top might not be so straightforward. Anyway, that's an answer to Mark's "how do you determine what got signed?" question.
To Mark's second question, the idea is that the final receiver would get two signatures: The first was added by the author and signed the author-generated content, and the second was added by the MLM and included the author content plus a footer in such a way that the two are separated in a structured way (MIME boundaries) in contrast to DKIM's simple (and deprecated) "l=" tag. The MLM signed the whole thing, while the author signed only the original content. We like to say that DKIM allows a domain to take some responsibility for a message and its content; you could then determine for which parts each of the two parties is responsible. Translate this into the DMARC world: I could now tell that this message was signed by the author, and this added part was signed by an MLM which probably means the MLM added the extra part, which contains only text, so now DKIM passes, and DMARC is satisfied.
A "real world" example: I'm A at a domain with a "reject" DMARC policy, mailman's running at B, and subscriber C runs at a domain that checks DMARC inbound. I send mail to a list at B, and sign it with my domain's signature; the MLM at B appends a footer in the MIME way and re-signs a message using this list-aware canonicalization; C confirms that (1) most of the content is signed by A, (2) one added text/plain or text/html part was added by B, (3) B doesn't have a scary reputation or isn't blacklisted, and (4) B's signature covered the added part; C considers that a DKIM pass for A, which is "aligned" (in DMARC terms), and thus DMARC is satisfied.
In theory, to deal with the non-MIME case, we could invoke DKIM's "l=" tag, but it would be an understatement to say the community would be pretty allergic to that idea.
-MSK
participants (3)
-
Daniel Kahn Gillmor
-
Murray S. Kucherawy
-
Stephen J. Turnbull