[Mailman-Users] Executive summary of DMARC issues

[Stephen J. Turnbull]

Gary Algier writes:

 > I have been following the discussion of the DMARC issues and Mailman's 
 > attempts to live with it.  I was wondering if anyone has an "Executive 
 > Summary" of the DMARC issue in a general sense.

How about the following:

DMARC is a set of protocols for Internet mail that are used to by
participating hosts to exchange information about mail traffic.  This
information is primarily useful for fighting email fraud, including
spam and phishing.  The basic procedure is that recipient hosts which
participate in the protocol check whether the apparent sending domain
(the domain that appears in the address in "From" header field)
participates, using a query to the DNS defined by the DMARC protocol.
If so, the recipient host will send to that domain some information
about mail which purports to be from the sending host, but fails to
validate.

The validation refers specifically to, and only to, the domain in the
"From" header field.  The DMARC protocol itself does not validate the
mailbox (user), although some originating hosts may do so.  Nor can it
guarantee that the user's account hasn't been used fraudulently.

For participants in the DMARC protocol, valid mail provides a strong
guarantee that the domain of the address in the "From" header field
is authentic, with some strong restrictions required to ensure success
of validation.  These restrictions include:

(1) When authentic mail is delivered directly from the sending host to
    the recipient, validation will succeed.

(2) When authentic mail is forwarded verbatim by all intervening
    Internet hosts (including mailing lists and user mailbox
    forwarding), validation will succeed.

Authentic mail may fail to validate in many common circumstances,
where a forwarding host alters certain headers or any part of the body
of the message.  This may occur for forwarding hosts which add
advertisements to the footer or the like, or if a forwarding host
reformats the body (it shouldn't do that, but some do).  There are
also restrictions on some parts of the mail.  For example, the "From"
header field must contain exactly one address (multiple author
addresses are permitted by the mail standards and are occasionally
useful, though rarely seen nowadays).

Failure to validate is also frequent on mailing lists.  Common
transformations that *will* cause mail distributed by a mailing list
to fail validation include (but are not limited to)

- Adding a list tag or sequence number to the Subject header field.

- Adding a footer, typically containing unsubscribe information for
  public discussion lists or legal disclaimers for corporate lists.

- Removal of banned content, such as executable file attachments or
  excessively large attachments.

Because many common practices with Internet mail can cause authentic
mail to fail DMARC validation, although mail which successfully
validates may be trusted to be from the domain in the address in the
"From" field, failure to validate in general *does not* imply lack of
authenticity.

Under some circumstances, a sending domain may be willing to severely
restrict its users' use of their addresses.  For example, they may not
be allowed to post to mailing lists, and recipients may be advised to
*always* keep their addresses updated to the domain where they
actually read their mail.  This is common practice with banks and
other financial institutions.  Such organizations can be reasonably
sure that mail which fails to validate is "fraudulent" (although the
intent may be merely mischievious rather than truly felonious).  They
may take advantage of a further feature of the DMARC protocol, the
"policy", which is the action that the sending domain suggests that
recipients take with mail that appears to be from that domain but
fails validation.

The normal policy is "none", that is, the recipient should deliver the
mail as though the apparent sending domain did not participate in
DMARC.  (Of course, if the mail fails spam or virus checks, it would
be quarantined or discarded as usual.)  The second level is
"quarantine", which means that the mail should not be delivered
directly to the user's mailbox in the ordinary way, for fear of
fraudulent use of the sending address (typically phishing).  The third
level is "reject", which means that the sender recommends that the
mail not be delivered at all, to completely prevent fraudulent misuse
of their domain.

Unfortunately, unless the sending domain has truly draconian policies
about the use of addresses in that domain, use of "quarantine" or
"reject" makes it virtually certain that some authentic mail will not
be read in a timely fashion or (in the case of "reject") never
delivered at all.

The "reject" policy may also result in denial of services to third
parties (one common case is having delivery disabled from a mailing
list, or even being unsubscribed), due to rejected mail being
"bounced" back to the forwarding host.  At present we know of *no*
hosts which distinguish DMARC bounces from other delivery failures.
Therefore they are treated as a permanent delivery failure to the
recipient due to problems at the recipient's host, rather than due to
a policy of the sender.  As a consequence, those unrelated recipients
may be dropped from distribution lists.

Editorial comments:

In some cases (where the mail validates according the DKIM, one of the
protocols used in DMARC validation), the mail can be trusted not to
have been altered in any way by a "man in the middle".

It is entirely up to the recipient whether to honor the published
policy or not.  (GMail, for example, does not -- it treats the policy
as advisory and makes case by case decisions, allowing most list mail
to be delivered despite failing DMARC validation.)  However, because
the original use of policy was envisioned to be banks vulnerable to
phishing (not large freemail providers incapable of protecting their
users' address books), most recipients do honor the policy published
by sending domains.

The action of Yahoo! and AOL in publishing policies of "reject"
basically amounts to deciding that they do not care if their users'
mail can be reliably delivered.  Nevertheless, Mailman is committed to
providing mitigation options for mailing list owners that improve the
reliability of mail delivery for posters from those domains, and
protect other subscribers from the (unintentional) denial of service
attacks being conducted by Yahoo! and AOL.

I believe the word "attack" is justified because Yahoo! and AOL are
now aware that their policies have the effect of denying service to
third parties (even if not intended by them), but they continue to
implement them.  It seems likely to me that as mailing lists and
others implement workarounds that bypass DMARC validation, these
workarounds will be mimicked by spam and phishing mails to bypass
DMARC validation, and in response AOL and Yahoo! will take further
action detrimental to the reliability of everyone's mail service, not
just their own users'.

 > Of course if someone says that the current MS365 implementation has
 > addressed this, then that's a different (unfortunate) story.

Seems unlikely to me (this isn't a new variant of a traditional
problem to be addressed by database updates like virus checking),
although they probably will roll out something within a couple months
(counting from April 22).  Nevertheless I doubt it will be as flexible
as Mailman already is (and most likely will be tuned to protect
Hotmail users if Hotmail should decide to use the "reject" policy).

 > Nielsen's First Law of Computer Manuals:
 >      People don't read documentation voluntarily.

How (sadly) apropos!

HTH

Steve