Issues with a DMARC record leading to message being shunted
![](https://secure.gravatar.com/avatar/267565c6ab7816fe29beedf9a9cbcd44.jpg?s=120&d=mm&r=g)
Hi!
Recently a mail was not being distributed via our MM3 instance; I searched the logs and found:
Dec 22 04:54:27 2023 (24) ACCEPT: <1d431542-b386-467b-8bf2-305f19ee7eb4@somenet.org> Dec 22 04:54:28 2023 (28) Uncaught runner exception: list index out of range Dec 22 04:54:28 2023 (28) Traceback (most recent call last): File "/usr/lib/python3.10/site-packages/mailman/core/runner.py", line 179, in _one_iteration self._process_one_file(msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/core/runner.py", line 272, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/runners/pipeline.py", line 37, in _dispose process(mlist, msg, msgdata, pipeline) File "/usr/lib/python3.10/site-packages/mailman/core/pipelines.py", line 53, in process handler.process(mlist, msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/handlers/validate_authenticity.py", line 125, in process authenticate(msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/utilities/retry.py", line 44, in f_retry return f(*args, **kwargs) File "/usr/lib/python3.10/site-packages/mailman/handlers/validate_authenticity.py", line 93, in authenticate auth_result = authenticate_message( File "/usr/lib/python3.10/site-packages/authheaders/__init__.py", line 395, in authenticate_message dmarc_result = check_dmarc(msg, spf_result, dkim_result, dnsfunc=dnsfunc, psddmarc=psddmarc) File "/usr/lib/python3.10/site-packages/authheaders/__init__.py", line 343, in check_dmarc result, result_comment, from_domain, policy = dmarc_per_from(from_domain, spf_result, dkim_result, dnsfunc, psddmarc) File "/usr/lib/python3.10/site-packages/authheaders/__init__.py", line 90, in dmarc_per_from record, orgdomain = receiver_record(from_domain) File "/usr/lib/python3.10/site-packages/authheaders/dmarc_lookup.py", line 117, in receiver_record retval = lookup_receiver_record(hostSansDmarc, dnsfunc) File "/usr/lib/python3.10/site-packages/authheaders/dmarc_lookup.py", line 92, in lookup_receiver_record tags = answer_to_dict(str(result)) File "/usr/lib/python3.10/site-packages/authheaders/dmarc_lookup.py", line 42, in answer_to_dict retval = {t[0].strip().lower(): t[1].strip().lower() for t in rawTags} File "/usr/lib/python3.10/site-packages/authheaders/dmarc_lookup.py", line 42, in <dictcomp> retval = {t[0].strip().lower(): t[1].strip().lower() for t in rawTags} IndexError: list index out of range
Dec 22 04:54:28 2023 (28) SHUNTING: 1703220868.1418982+b27f5db154375fcd2c418d467172e4aad0d3d57a
So the message has been shunted, due to this error. Soemthing wit DMARC and tags. So I checked the DMARC record:
The DMARC record for somenet.org is "v=DMARC1;p=none;pct=100;rua=mailto:postmaster@somenet.org;mailto:postmaster@somenet.org;ri=3600;fo=1;" which is syntactically incorrect. The extra ";mailto:postmaster@somenet.org" is wrong, I guess the tag "ruf=" is missing here!
It should probably read "v=DMARC1;p=none;pct=100;rua=mailto:postmaster@somenet.org;ruf=mailto:postmaster@somenet.org;ri=3600;fo=1;"
So, should mm3 somehow "catch" this error somehow? I wonder how a syntactically incorrect DMARC record should be handled at all (ignore? best effort?)
-- Ralf Hildebrandt Charité - Universitätsmedizin Berlin Geschäftsbereich IT | Abteilung Netz | Netzwerk-Administration Invalidenstraße 120/121 | D-10115 Berlin
Tel. +49 30 450 570 155 ralf.hildebrandt@charite.de https://www.charite.de
![](https://secure.gravatar.com/avatar/e2371bef92eb40cd7c586e9f2cc75cd8.jpg?s=120&d=mm&r=g)
Ralf Hildebrandt via Mailman-Users writes:
Seems very likely.
It should probably read "v=DMARC1;p=none;pct=100;rua=mailto:postmaster@somenet.org;ruf=mailto:postmaster@somenet.org;ri=3600;fo=1;"
So, should mm3 somehow "catch" this error somehow?
Yes. The parser is in the authheaders package, a 3rd-party package on PyPI. I'll get in touch with the maintainers, but given the season I wouldn't bet on a release soon. Of course you may get lucky with an upgrade if you're not up-to-date (v0.15.3). Upgrade should be safe, the maintainers are reliable authors in this field.
The rest of this post is for RFC nerds. :-)
I wonder how a syntactically incorrect DMARC record should be handled at all
Per RFC 7489, Sec. 6.3:
A DMARC policy record MUST comply with the formal specification found
in Section 6.4 in that the "v" and "p" tags MUST be present and MUST
appear in that order. Unknown tags MUST be ignored. Syntax errors
in the remainder of the record SHOULD be discarded in favor of
default values (if any) or ignored outright.
I'm pretty sure the best strategy is to divide the string on ";", check that the first component is exactly "v=DMARC1", the second is a "p=..." specification. If either of those fails, the whole record should be ignored (but the receiver can go ahead and do DMARC processing and make its own decisions based on the result).
After that, any component that is not of the form "tag=value" for a tag that is defined in the RFC and a value that is syntactically valid for the tag should be ignored but parsing continues for the remaining components..
There is no default for ruf, so IMO you must ignore it:
ruf: ... If not provided, Mail Receivers MUST NOT generate
failure reports.
Although the evidence for ruf=mailto:postmaster@somenet.org is really strong, given it's the same URL as rua and fo=1; best effort might apply.
(ignore?
The whole policy? Definitely not, unless the "v" and "p" tags are missing or syntactically incorrect. Of course it's optional for receivers to participate, but if you participate, best effort is indicated (Sec. 6):
A Mail Receiver implementing the DMARC mechanism SHOULD make a
best-effort attempt to adhere to the Domain Owner's published DMARC
policy when a message fails the DMARC test.
![](https://secure.gravatar.com/avatar/267565c6ab7816fe29beedf9a9cbcd44.jpg?s=120&d=mm&r=g)
I'm running 0.15.1, but that doesn't seem to be covered by the changes for .2 & .3
-- Ralf Hildebrandt Charité - Universitätsmedizin Berlin Geschäftsbereich IT | Abteilung Netz | Netzwerk-Administration Invalidenstraße 120/121 | D-10115 Berlin
Tel. +49 30 450 570 155 ralf.hildebrandt@charite.de https://www.charite.de
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 12/23/23 12:18 AM, Ralf Hildebrandt via Mailman-Users wrote:
Please follow up to the thread at https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/A...
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
![](https://secure.gravatar.com/avatar/e2371bef92eb40cd7c586e9f2cc75cd8.jpg?s=120&d=mm&r=g)
Ralf Hildebrandt via Mailman-Users writes:
Seems very likely.
It should probably read "v=DMARC1;p=none;pct=100;rua=mailto:postmaster@somenet.org;ruf=mailto:postmaster@somenet.org;ri=3600;fo=1;"
So, should mm3 somehow "catch" this error somehow?
Yes. The parser is in the authheaders package, a 3rd-party package on PyPI. I'll get in touch with the maintainers, but given the season I wouldn't bet on a release soon. Of course you may get lucky with an upgrade if you're not up-to-date (v0.15.3). Upgrade should be safe, the maintainers are reliable authors in this field.
The rest of this post is for RFC nerds. :-)
I wonder how a syntactically incorrect DMARC record should be handled at all
Per RFC 7489, Sec. 6.3:
A DMARC policy record MUST comply with the formal specification found
in Section 6.4 in that the "v" and "p" tags MUST be present and MUST
appear in that order. Unknown tags MUST be ignored. Syntax errors
in the remainder of the record SHOULD be discarded in favor of
default values (if any) or ignored outright.
I'm pretty sure the best strategy is to divide the string on ";", check that the first component is exactly "v=DMARC1", the second is a "p=..." specification. If either of those fails, the whole record should be ignored (but the receiver can go ahead and do DMARC processing and make its own decisions based on the result).
After that, any component that is not of the form "tag=value" for a tag that is defined in the RFC and a value that is syntactically valid for the tag should be ignored but parsing continues for the remaining components..
There is no default for ruf, so IMO you must ignore it:
ruf: ... If not provided, Mail Receivers MUST NOT generate
failure reports.
Although the evidence for ruf=mailto:postmaster@somenet.org is really strong, given it's the same URL as rua and fo=1; best effort might apply.
(ignore?
The whole policy? Definitely not, unless the "v" and "p" tags are missing or syntactically incorrect. Of course it's optional for receivers to participate, but if you participate, best effort is indicated (Sec. 6):
A Mail Receiver implementing the DMARC mechanism SHOULD make a
best-effort attempt to adhere to the Domain Owner's published DMARC
policy when a message fails the DMARC test.
![](https://secure.gravatar.com/avatar/267565c6ab7816fe29beedf9a9cbcd44.jpg?s=120&d=mm&r=g)
I'm running 0.15.1, but that doesn't seem to be covered by the changes for .2 & .3
-- Ralf Hildebrandt Charité - Universitätsmedizin Berlin Geschäftsbereich IT | Abteilung Netz | Netzwerk-Administration Invalidenstraße 120/121 | D-10115 Berlin
Tel. +49 30 450 570 155 ralf.hildebrandt@charite.de https://www.charite.de
![](https://secure.gravatar.com/avatar/56f108518d7ee2544412cc80978e3182.jpg?s=120&d=mm&r=g)
On 12/23/23 12:18 AM, Ralf Hildebrandt via Mailman-Users wrote:
Please follow up to the thread at https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/A...
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (3)
-
Mark Sapiro
-
Ralf Hildebrandt
-
Stephen J. Turnbull