DMARC mitigation failing on one list
We have DMARC "munge from" configured on all mailing lists on lists.freebsd.org.
This week, I learned that one of our lists is not actually munging though.
I've done a config_list -o and compared the output to a list that does
munge correctly and I'm not seeing anything that would explain the
problem.
When a poster from a DMARC-crippled domain tries to post to this list, I see a line in vette that the DMARC policy has been found. There are no messages in the error log.
Any idea what could explain this? And how I could go about fixing it?
Many thanks.
Philip
-- Philip Paeps Senior Reality Engineer Alternative Enterprises
On 2021-01-24 14:08:28 (+0800), Philip Paeps wrote:
We have DMARC "munge from" configured on all mailing lists on lists.freebsd.org.
This week, I learned that one of our lists is not actually munging though.
I've done a
config_list -oand compared the output to a list that does munge correctly and I'm not seeing anything that would explain the problem.When a poster from a DMARC-crippled domain tries to post to this list, I see a line in vette that the DMARC policy has been found. There are no messages in the error log.
Any idea what could explain this? And how I could go about fixing it?
Another data point: to experiment, I added ^.*@yahoo\.com$ to dmarc_moderation_addresses for this list. It still doesn't munge.
Every other list on our system "just works". This one is refusing to cooperate.
Unfortunately, this is a live list with several thousand subscribers with various combinations of options and passwords configured so I can't simply blow the list away and create it again.
Can anyone think of something I may be overlooking?
Many thanks. Philip
-- Philip Paeps Senior Reality Engineer Alternative Enterprises
On 1/24/21 7:46 PM, Philip Paeps wrote:
On 2021-01-24 14:08:28 (+0800), Philip Paeps wrote:
We have DMARC "munge from" configured on all mailing lists on lists.freebsd.org.
This week, I learned that one of our lists is not actually munging though. ...
Another data point: to experiment, I added ^.*@yahoo\.com$ to dmarc_moderation_addresses for this list. It still doesn't munge.
Every other list on our system "just works". This one is refusing to cooperate.
I suspect this list has a pipeline attribute defined[1] and the list's pipeline does not include SpamDetect or possibly CookHeaders.
What does
bin/dumpdb lists/LISTNAME/config | grep -A24 pipeline
show?
[1] See https://bazaar.launchpad.net/~mailman-coders/mailman/2.1/view/head:/Mailman/... through line 144.
Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 2021-01-25 13:48:38 (+0800), Mark Sapiro wrote:
On 1/24/21 7:46 PM, Philip Paeps wrote:
On 2021-01-24 14:08:28 (+0800), Philip Paeps wrote:
We have DMARC "munge from" configured on all mailing lists on lists.freebsd.org.
This week, I learned that one of our lists is not actually munging though. ...
Another data point: to experiment, I added ^.*@yahoo\.com$ to dmarc_moderation_addresses for this list. It still doesn't munge.
Every other list on our system "just works". This one is refusing to cooperate.
I suspect this list has a pipeline attribute defined[1] and the list's pipeline does not include SpamDetect or possibly CookHeaders.
What does
bin/dumpdb lists/LISTNAME/config | grep -A24 pipeline
show?
Thanks for the tip! This list does have a pipeline attribute.
'pipeline': [ 'DeDuper',
'SpamDetect',
'Approve',
'Replybot',
'Moderate',
'Hold',
'MimeDel',
'Scrubber',
'Emergency',
'Tagger',
'CalcRecips',
'AvoidDuplicates',
'Cleanse',
'CleanseDKIM',
'CookHeaders',
'ToDigest',
'ToArchive',
'ToUsenet',
'AfterDelivery',
'Acknowledge',
'ToOutgoing'],
It seems to have both SpamDetect and CookHeaders. Our other mailing lists do not have a pipeline attribute defined though. Suspiciously, this pipeline is missing the WrapMessage stage present in the GLOBAL_PIPELINE. I wonder if that could be the problem?
I'll try to remove the pipeline attribute and see if that works.
Many thanks for the pointer.
Philip
-- Philip Paeps Senior Reality Engineer Alternative Enterprises
On 2021-01-25 14:02:55 (+0800), Philip Paeps wrote:
On 2021-01-25 13:48:38 (+0800), Mark Sapiro wrote:
On 1/24/21 7:46 PM, Philip Paeps wrote:
On 2021-01-24 14:08:28 (+0800), Philip Paeps wrote:
We have DMARC "munge from" configured on all mailing lists on lists.freebsd.org.
This week, I learned that one of our lists is not actually munging though. ...
Another data point: to experiment, I added ^.*@yahoo\.com$ to dmarc_moderation_addresses for this list. It still doesn't munge.
Every other list on our system "just works". This one is refusing to cooperate.
I suspect this list has a pipeline attribute defined[1] and the list's pipeline does not include SpamDetect or possibly CookHeaders.
What does
bin/dumpdb lists/LISTNAME/config | grep -A24 pipeline
show?
Thanks for the tip! This list does have a
pipelineattribute.[...]
It seems to have both SpamDetect and CookHeaders. Our other mailing lists do not have a pipeline attribute defined though. Suspiciously, this pipeline is missing the WrapMessage stage present in the GLOBAL_PIPELINE. I wonder if that could be the problem?
I'll try to remove the pipeline attribute and see if that works.
Many thanks for the pointer.
That did work. Thank you! This had me stumped for a couple of days.
Philip
-- Philip Paeps Senior Reality Engineer Alternative Enterprises
On 1/24/21 10:02 PM, Philip Paeps wrote:
Thanks for the tip! This list does have a
pipelineattribute.'pipeline': [ 'DeDuper', 'SpamDetect', 'Approve', 'Replybot', 'Moderate', 'Hold', 'MimeDel', 'Scrubber', 'Emergency', 'Tagger', 'CalcRecips', 'AvoidDuplicates', 'Cleanse', 'CleanseDKIM', 'CookHeaders', 'ToDigest', 'ToArchive', 'ToUsenet', 'AfterDelivery', 'Acknowledge', 'ToOutgoing'],It seems to have both SpamDetect and CookHeaders. Our other mailing lists do not have a pipeline attribute defined though. Suspiciously, this pipeline is missing the WrapMessage stage present in the GLOBAL_PIPELINE. I wonder if that could be the problem?
Yes, the absence of WrapMessage is the problem. I forgot that WrapMessage also is the piece that actually replaces the original From:. SpamDetect is where we determine that the message needs DMARC mitigation, and CookHeaders is where we figure what the munged From: should look like, but CookHeaders only puts in in the messages metadata and WrapMessage actually does the replacement.
If you were to add WrapMessge, the only difference between this and GLOBAL_PIPELINE is this one added DeDuper at the beginning. Since I don't know what that does, I don't know if it's important or not, but if it is, you could restore the list's pipeline with the addition of WrapMessage.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 2021-01-25 23:52:53 (+0800), Mark Sapiro wrote:
On 1/24/21 10:02 PM, Philip Paeps wrote:
Thanks for the tip! This list does have a
pipelineattribute.[...]
It seems to have both SpamDetect and CookHeaders. Our other mailing lists do not have a pipeline attribute defined though. Suspiciously, this pipeline is missing the WrapMessage stage present in the GLOBAL_PIPELINE. I wonder if that could be the problem?
Yes, the absence of WrapMessage is the problem. I forgot that WrapMessage also is the piece that actually replaces the original
From:. SpamDetect is where we determine that the message needs DMARC mitigation, and CookHeaders is where we figure what the munged From: should look like, but CookHeaders only puts in in the messages metadata and WrapMessage actually does the replacement.
Thanks for the extra context. I'm beginning to understand how this works now.
If you were to add WrapMessge, the only difference between this and GLOBAL_PIPELINE is this one added DeDuper at the beginning. Since I don't know what that does, I don't know if it's important or not, but if it is, you could restore the list's pipeline with the addition of WrapMessage.
The DeDuper is a local addition we have (and it exists in our
GLOBAL_PIPELINE). It keeps a cache of recently seen message-ids on the
list and discards duplicates. This is in place because our lists can be
addressed both as freebsd-current@freebsd.org and current@freebsd.org.
When someone does reply-all to a message with an original To: header of
current@freebsd.org, two copies of the message arrive at the list.
Philip
-- Philip Paeps Senior Reality Engineer Alternative Enterprises
On 1/25/21 1:18 PM, Philip Paeps wrote:
Thanks for the extra context. I'm beginning to understand how this works now.
If you were to add WrapMessge, the only difference between this and GLOBAL_PIPELINE is this one added DeDuper at the beginning. Since I don't know what that does, I don't know if it's important or not, but if it is, you could restore the list's pipeline with the addition of WrapMessage.
The DeDuper is a local addition we have (and it exists in our GLOBAL_PIPELINE). It keeps a cache of recently seen message-ids on the list and discards duplicates. This is in place because our lists can be addressed both as freebsd-current@freebsd.org and current@freebsd.org. When someone does reply-all to a message with an original To: header of current@freebsd.org, two copies of the message arrive at the list.
OK. So the correct pipeline for this list with WrapMessage included is the same as your GLOBAL_PIPELINE, so just deleting the list specific pipeline is the correct solution.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Sun, 24 Jan 2021 21:48:38 -0800 Mark Sapiro <mark@msapiro.net> wrote:
On 1/24/21 7:46 PM, Philip Paeps wrote:
On 2021-01-24 14:08:28 (+0800), Philip Paeps wrote:
We have DMARC "munge from" configured on all mailing lists on lists.freebsd.org.
This week, I learned that one of our lists is not actually munging though.
...Another data point: to experiment, I added ^.*@yahoo\.com$ to dmarc_moderation_addresses for this list. It still doesn't munge.
Every other list on our system "just works". This one is refusing to cooperate.
I suspect this list has a pipeline attribute defined[1] and the list's pipeline does not include SpamDetect or possibly CookHeaders.
What does
bin/dumpdb lists/LISTNAME/config | grep -A24 pipeline
show?
[1] See https://bazaar.launchpad.net/~mailman-coders/mailman/2.1/view/head:/Mailman/... through line 144.
Good Morning,
I couldn't help noticing this thread since I am dealing with DMARC issues at the moment. May I use this forum to ask: Which approach is better in any given scenario between "Munge From" and "Wrap Message" (i.e. a little insight on when one is a better choice over the other)?
Regards Onyeibo
On 2021-01-25 18:39:15 (+0800), Onyeibo wrote:
On Sun, 24 Jan 2021 21:48:38 -0800 Mark Sapiro <mark@msapiro.net> wrote:
On 1/24/21 7:46 PM, Philip Paeps wrote:
On 2021-01-24 14:08:28 (+0800), Philip Paeps wrote:
We have DMARC "munge from" configured on all mailing lists on lists.freebsd.org.
This week, I learned that one of our lists is not actually munging though. ...
Another data point: to experiment, I added ^.*@yahoo\.com$ to dmarc_moderation_addresses for this list. It still doesn't munge.
Every other list on our system "just works". This one is refusing to cooperate.
I suspect this list has a pipeline attribute defined[1] and the list's pipeline does not include SpamDetect or possibly CookHeaders.
What does
bin/dumpdb lists/LISTNAME/config | grep -A24 pipeline
show?
[1] See https://bazaar.launchpad.net/~mailman-coders/mailman/2.1/view/head:/Mailman/... through line 144.
Good Morning,
I couldn't help noticing this thread since I am dealing with DMARC issues at the moment. May I use this forum to ask: Which approach is better in any given scenario between "Munge From" and "Wrap Message" (i.e. a little insight on when one is a better choice over the other)?
In my experience, "wrap message" has the same downside as digests ... people will reply to the wrapper rather than the wrapped message (as they would reply to the digest rather than to an individual message within) and break the thread for everyone else.
Very few mail clients seem to support replying to nested messsages.
Munging appears to strike the right balance between frustration and functionality.
At the end of the day, DMARC still breaks the mailing list experience.
We can only try to mitigate the damage.
Philip
-- Philip Paeps Senior Reality Engineer Alternative Enterprises
On 1/25/21 5:02 AM, Philip Paeps wrote:
On 2021-01-25 18:39:15 (+0800), Onyeibo wrote:
I couldn't help noticing this thread since I am dealing with DMARC issues at the moment. May I use this forum to ask: Which approach is better in any given scenario between "Munge From" and "Wrap Message" (i.e. a little insight on when one is a better choice over the other)?
In my experience, "wrap message" has the same downside as digests ... people will reply to the wrapper rather than the wrapped message (as they would reply to the digest rather than to an individual message within) and break the thread for everyone else.
Very few mail clients seem to support replying to nested messsages.
Munging appears to strike the right balance between frustration and functionality.
I agree with everything Philip says. Wrap message effectively sends each message as a MIME format digest containing the one message. Just as some people, mostly depending on their choice of MUA, will prefer plain text digests over MIME, and vice versa, some will prefer a wrapped message over a munged from, and vice versa.
In short, which is better depends on the characteristics of the list and its members, but in general, from munging seems the better choice, although arguably and in the opinion of the Mailman developers, munging the From: header in this way makes the message non-compliant with RFC 5322 and its predecessors.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (3)
-
Mark Sapiro -
Onyeibo -
Philip Paeps