Re: [Mailman-Developers] MIME footers
Hi Murray--
On Fri 2015-02-27 14:46:40 -0500, Murray S. Kucherawy wrote:
Sorry, by "sign" I meant "add a footer". I probably said "sign" because this is related to some DKIM work I've been planning, and the morning's caffeine was already wearing off.
:)
Thanks for that detailed answer (and Barry for his followup). It's precisely what I was looking for.
How absurd would it be to propose a flag for Mailman that would take your first case (non-MIME, or single-part text/plain) and convert it to a multipart/mixed with a child part of the original text/plain, and then apply the algorithm you have?
I'm not one to make any decisions for mailman, but this sounds pretty reasonable to me. In fact, the only argument against ever having such a switch *not* set is that non-MIME-aware text-only MUAs will now see some crufty MIME boundaries in the message that they didn't see before.
I'd bet the number of users of non-MIME-aware text-only MUAs receiving mail from mailman that are incapable of wading through a bit of extra MIME boundary info on otherwise text/plain messages is vanishingly small. Most of those folks are probably at least used to seeing it as a result of the overwhelming predominance of multipart/alternative on today's SMTP network.
The impetus here is DKIM survivability across lists. Suppose we had a DKIM canonicalization that was MIME-aware, so that it could sign the specific MIME parts or sets of parts. That signature would fail on the message as a whole -- with the footer part added -- but could in theory pass if an appended part were omitted from canonicalization. To put it in context, suppose there were a DKIM canonicalization where the signer signed (using your examples) the CDE message; the receiver gets FGHI which fails, but also has enough information to know that merely verifying FGH will pass; it then knows that FGH was "legitimate" and I was added post-signing, and may or may not be "safe" (for some value thereof) content.
What I'm worried about with such a design is the trivial text/plain message. Obviously merely appending the footer destroys any hope of validating only the original content. We'd have to entertain the idea that Mailman would make the simple message into a multipart/mixed + text/plain, then append the footer part and sign that; the verifier would drop the footer and then strip off the MIME to see if it can verify the original signature that way. That seems like its easy to get wrong, though it's likely to be a very common case.
right, the MIME boundaries provide the parsing framework to enable this kind of sub-part DKIM-verification.
One issue this process brings up is that it's now necessary to treat pretty much every part of the message as though it is multipart/signed -- that is, it needs to be handled bytewise opaquely, on pain of breaking the DKIM header.
We already know that there are tools that do things like re-encode messages from base64 to q-p, or change character sets, or just re-flow long lines (FSVO "long") in the mime boundary subheaders. They're not supposed to do this with multipart/signed messages, because the RFCs point out that opaque handling is required, but they sometimes break them anyway. I imagine the same risk would now apply to all DKIM-signed messages using your scheme.
With that caveat in mind, it's clear that without your proposal, all DKIM-signatures of messages with footers will break anyway. So this does seem to be a strict improvement, even if there are a few corner cases that aren't fully covered.
--dkg
Daniel Kahn Gillmor writes:
I'd bet the number of users of non-MIME-aware text-only MUAs receiving mail from mailman that are incapable of wading through a bit of extra MIME boundary info on otherwise text/plain messages is vanishingly small.
Not vanishing. I'll add my testimony to Mark's that we see complaints related to added MIME parts (mostly from people who read the text/html alternative when it is kept, I think, but there are some people still using broken old MUAs, and the real hurt is when somebody's boss's MUA doesn't display the legal BS at the bottom -- neither the boss nor his lawyers are interested in "but we can't do anything about broken MUAs").
Most of those folks are probably at least used to seeing it as a result of the overwhelming predominance of multipart/alternative on today's SMTP network.
For some reason, no, that doesn't seem to be the case.
OK, this time I was properly caffeinated, but that also meant I had a short attention span. :-) Sorry for the long delay replying.
On Fri, Feb 27, 2015 at 12:55 PM, Daniel Kahn Gillmor wrote: One issue this process brings up is that it's now necessary to treat
pretty much every part of the message as though it is
multipart/signed -- that is, it needs to be handled bytewise opaquely,
on pain of breaking the DKIM header. We already know that there are tools that do things like re-encode
messages from base64 to q-p, or change character sets, or just re-flow
long lines (FSVO "long") in the mime boundary subheaders. They're not
supposed to do this with multipart/signed messages, because the RFCs
point out that opaque handling is required, but they sometimes break
them anyway. I imagine the same risk would now apply to all DKIM-signed
messages using your scheme. The proposal I cobbled together (which is now in an expired IETF draft; I
need to post an update so it's visible again, late this month probably)
actually required that the canonicalized form of a MIME part is the fully
decoded form. That way, 7bit, 8bit, q-p and base64 re-encodings would have
no effect on the produced hash, and the signature would survive.
Theoretically. The tricky part is going to be the question of how to sign something that's
not MIME, and then how to handle stuff like MIME preambles and postambles.
There's going to be a lot of fussing over what various MUAs do with such
content, and whether that should be signed, and so on. The best I think we
can do is sign the actual part content, and be exceptionally clear about
what leading and trailing spaces get included or omitted, etc. One byte of
ambiguity and you're toast. That said, DKIM itself did a pretty good job
of this, so we have that to use as a model. And, if I may reply to some other points made on this thread: As to the question of crusty MUAs, I'm inclined to agree with the
"vanishingly small" characterization, except that as is typical in these
situations, it's a rather vocal small. I know for example that alpine
shows the preamble even though one could argue that it's not a usable part
of a MIME message. Certainly the goal is to be as incremental and
non-disruptive as possible, so I'd be inclined to do what someone
suggested, which is to make it a flag (a list flag, or maybe a user flag),
defaulting to something sane. But at some point we have to give up here, I
feel; stuff like this becomes an attack directly on the MUA, and I would
argue that there's only so much an MLM can (or should) do to protect users
from crappy software. I had forgotten about message headers (i.e., prepended text). Are those
common? I had thought pretty much everyone uses footers only. It's
certainly the case that this proposal only deals well with footers. The
specific algorithm is to construct a MIME tree and sign parts of it;
specifically, sign all of it, and then verify all of what you get first.
The changed part would be in a new leaf, which is easy to exclude from hash
generation. Excluding a part at or near the top might not be so
straightforward. Anyway, that's an answer to Mark's "how do you determine
what got signed?" question. To Mark's second question, the idea is that the final receiver would get
two signatures: The first was added by the author and signed the
author-generated content, and the second was added by the MLM and included
the author content plus a footer in such a way that the two are separated
in a structured way (MIME boundaries) in contrast to DKIM's simple (and
deprecated) "l=" tag. The MLM signed the whole thing, while the author
signed only the original content. We like to say that DKIM allows a domain
to take some responsibility for a message and its content; you could then
determine for which parts each of the two parties is responsible.
Translate this into the DMARC world: I could now tell that this message was
signed by the author, and this added part was signed by an MLM which
probably means the MLM added the extra part, which contains only text, so
now DKIM passes, and DMARC is satisfied. A "real world" example: I'm A at a domain with a "reject" DMARC policy,
mailman's running at B, and subscriber C runs at a domain that checks DMARC
inbound. I send mail to a list at B, and sign it with my domain's
signature; the MLM at B appends a footer in the MIME way and re-signs a
message using this list-aware canonicalization; C confirms that (1) most of
the content is signed by A, (2) one added text/plain or text/html part was
added by B, (3) B doesn't have a scary reputation or isn't blacklisted, and
(4) B's signature covered the added part; C considers that a DKIM pass for
A, which is "aligned" (in DMARC terms), and thus DMARC is satisfied. In theory, to deal with the non-MIME case, we could invoke DKIM's "l=" tag,
but it would be an understatement to say the community would be pretty
allergic to that idea. -MSK
participants (3)
-
Daniel Kahn Gillmor -
Murray S. Kucherawy -
Stephen J. Turnbull