"Message discarded" "spamdetect", why?
Hi, yesterday, I member tried multiple times to send a to a list, from a roundcube webmailer which is running on the same host as mailman itself. The message is in plain text without any attachments, no HTML either. Yet it was silently discarded by mailman. In /var/log/mailman/vette I find the message Jun 19 21:02:42 2020 (14835) Message discarded, msgid: <************@***>' list: ******, handler: SpamDetect The filter configuration for the list is: filter_mime_types = '' pass_mime_types = """multipart/mixed multipart/alternative text/plain""" filter_filename_extensions = """exe bat cmd com pif scr vbs cpl""" pass_filename_extensions = '' Can anyone say why mailman considered it to be spam? BTW, all messages are scanned by rspamd anyway, which does an outstanding job at spam detection, so mailman's rudimentary spam detection mechanism seems rather superfluous to me. Can't one just turn it off altogether? Cheers, Johannes
On 6/20/20 5:52 AM, Johannes Rohr wrote:
Hi,
yesterday, I member tried multiple times to send a to a list, from a roundcube webmailer which is running on the same host as mailman itself. The message is in plain text without any attachments, no HTML either. Yet it was silently discarded by mailman. In /var/log/mailman/vette I find the message
Jun 19 21:02:42 2020 (14835) Message discarded, msgid: <************@***>' list: ******, handler: SpamDetect
There are 3 reasons why SpamDetect can discard a message.
1) (not likely) The message contains a header with value matching a
header/regexp combination in mm_cfg.KNOWN_SPAMMERS which is empty by
default.
2) The message is from a sender in a domain publishing a DMARC policy of
reject or possibly quarantine and the list's Privacy options... ->
Sender filters -> dmarc_moderation_action is Discard.
3) The message matched a Spam Filter Rule in Privacy options... -> Spam
filters -> header_filter_rules with a Discard Action.
Check these things in your list configuration.
--
Mark Sapiro
Am 20.06.20 um 19:37 schrieb Mark Sapiro:
On 6/20/20 5:52 AM, Johannes Rohr wrote:
Hi,
yesterday, I member tried multiple times to send a to a list, from a roundcube webmailer which is running on the same host as mailman itself. The message is in plain text without any attachments, no HTML either. Yet it was silently discarded by mailman. In /var/log/mailman/vette I find the message
Jun 19 21:02:42 2020 (14835) Message discarded, msgid: <************@***>' list: ******, handler: SpamDetect
There are 3 reasons why SpamDetect can discard a message.
1) (not likely) The message contains a header with value matching a header/regexp combination in mm_cfg.KNOWN_SPAMMERS which is empty by default.
2) The message is from a sender in a domain publishing a DMARC policy of reject or possibly quarantine and the list's Privacy options... -> Sender filters -> dmarc_moderation_action is Discard.
Thanks, Marc, a lot for the explanation!!! Very helpful!! Now, for the messages that got rejected, none of the three reasons apply. However, in /var/log/mailman/error, I see the message I wrote about " DNSException: Unable to query DMARC policy for *** (_dmarc.***.org). The DNS operation timed out." with both the time and sender address matching the rejected message. So could it be that mailman treats a failed DMARC DNS query as if the domain in question had set its DMARC policy to "reject" or "quarantine"? And in our case, I feel that disabling this kind of checks altogether would make sense, given that all mail is parsed by rspamd before it is handed over to mailman. rspamd does an outstanding job, therefore, I don't think that there is a need for mailman to parse messages for spam. Cheers, Johannes
3) The message matched a Spam Filter Rule in Privacy options... -> Spam filters -> header_filter_rules with a Discard Action.
Check these things in your list configuration.
On 6/20/20 4:39 PM, Johannes Rohr wrote:
Am 20.06.20 um 19:37 schrieb Mark Sapiro:
On 6/20/20 5:52 AM, Johannes Rohr wrote:
Hi,
yesterday, I member tried multiple times to send a to a list, from a roundcube webmailer which is running on the same host as mailman itself. The message is in plain text without any attachments, no HTML either. Yet it was silently discarded by mailman. In /var/log/mailman/vette I find the message
Jun 19 21:02:42 2020 (14835) Message discarded, msgid: <************@***>' list: ******, handler: SpamDetect There are 3 reasons why SpamDetect can discard a message.
1) (not likely) The message contains a header with value matching a header/regexp combination in mm_cfg.KNOWN_SPAMMERS which is empty by default.
2) The message is from a sender in a domain publishing a DMARC policy of reject or possibly quarantine and the list's Privacy options... -> Sender filters -> dmarc_moderation_action is Discard. Thanks, Marc, a lot for the explanation!!! Very helpful!! Now, for the messages that got rejected, none of the three reasons apply. However, in /var/log/mailman/error, I see the message I wrote about " DNSException: Unable to query DMARC policy for *** (_dmarc.***.org). The DNS operation timed out." with both the time and sender address matching the rejected message. So could it be that mailman treats a failed DMARC DNS query as if the domain in question had set its DMARC policy to "reject" or "quarantine"?
And in our case, I feel that disabling this kind of checks altogether would make sense, given that all mail is parsed by rspamd before it is handed over to mailman. rspamd does an outstanding job, therefore, I don't think that there is a need for mailman to parse messages for spam.
Cheers,
Johannes
Marc can confirm, but I do believe that a failed DNS Query is treated as if the policy was reject, on the theory that it is better to mitigate in error than to not mitigate in error. Having a policy of discard would make that assumption not so good (it is probably more common to munge or wrap) -- Richard Damon
On 6/20/20 1:39 PM, Johannes Rohr wrote:
Am 20.06.20 um 19:37 schrieb Mark Sapiro:
On 6/20/20 5:52 AM, Johannes Rohr wrote:
Jun 19 21:02:42 2020 (14835) Message discarded, msgid: <************@***>' list: ******, handler: SpamDetect
There are 3 reasons why SpamDetect can discard a message.
1) (not likely) The message contains a header with value matching a header/regexp combination in mm_cfg.KNOWN_SPAMMERS which is empty by default.
2) The message is from a sender in a domain publishing a DMARC policy of reject or possibly quarantine and the list's Privacy options... -> Sender filters -> dmarc_moderation_action is Discard.
Thanks, Marc, a lot for the explanation!!! Very helpful!! Now, for the messages that got rejected, none of the three reasons apply. However, in /var/log/mailman/error, I see the message I wrote about " DNSException: Unable to query DMARC policy for *** (_dmarc.***.org). The DNS operation timed out." with both the time and sender address matching the rejected message. So could it be that mailman treats a failed DMARC DNS query as if the domain in question had set its DMARC policy to "reject" or "quarantine"?
Yes. That's exactly what it does. If it can't get an answer from DNS, it assumes that mitigation when not required is safer than not mitigating when required. Arguably, this is not correct if the dmarc_moderation_action is Discard or even Reject, but that's what it does.
And in our case, I feel that disabling this kind of checks altogether would make sense, given that all mail is parsed by rspamd before it is handed over to mailman. rspamd does an outstanding job, therefore, I don't think that there is a need for mailman to parse messages for spam.
The Handler that does this is SpamDetect, but it isn't doing the DMARC
checks for spam reasons. It's just that there were a few checks
including DMARC that needed to be checked early in the pipeline and
rather than creating a new handler for them, I added them to SpamDetect.
In your case, the message is discarded for DMARC policy reasons,
presumably because of the DNS timeout. Do you really want this list
configured to discard these messages?
--
Mark Sapiro
Am Samstag, den 20.06.2020, 13:59 -0700 schrieb Mark Sapiro:
[...]
In your case, the message is discarded for DMARC policy reasons, presumably because of the DNS timeout. Do you really want this list configured to discard these messages?
No, that's precisely what I want to avoid, but that's precisely what happened. Strangely though, not what happens every time a DNS query fails but it did happen some of the time.
Cheers,
Johannes
On 6/21/20 12:27 AM, jorohr@gmail.com wrote:
Am Samstag, den 20.06.2020, 13:59 -0700 schrieb Mark Sapiro:
[...]
In your case, the message is discarded for DMARC policy reasons, presumably because of the DNS timeout. Do you really want this list configured to discard these messages?
No, that's precisely what I want to avoid, but that's precisely what happened. Strangely though, not what happens every time a DNS query fails but it did happen some of the time.
The list it happens on has its Privacy options... -> Sender filters -> dmarc_moderation_action set to Discard.
If that is not the case and "SpamDetect" processing is discarding some but not all messages for the same list, then perhaps it is not DMARC but header_filter_rules responsible for the discard.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Am 21.06.20 um 18:15 schrieb Mark Sapiro:
The list it happens on has its Privacy options... -> Sender filters -> dmarc_moderation_action set to Discard.
It isn't. It is set o "0".
If that is not the case and "SpamDetect" processing is discarding some but not all messages for the same list, then perhaps it is not DMARC but header_filter_rules responsible for the discard.
It is also empty:
header_filter_rules = []
Cheers,
Johannes
On 6/23/20 6:12 AM, Johannes Rohr wrote:
Am 21.06.20 um 18:15 schrieb Mark Sapiro:
The list it happens on has its Privacy options... -> Sender filters -> dmarc_moderation_action set to Discard.
It isn't. It is set o "0".
If that is not the case and "SpamDetect" processing is discarding some but not all messages for the same list, then perhaps it is not DMARC but header_filter_rules responsible for the discard.
It is also empty:
header_filter_rules = []
These are settings for a list. Is that the list named in the Message discarded log entry?
There is also the global Defaults.py/mm_cfg.py setting KNOWN_SPAMMERS. If that is also empty, there is nothing that would cause SpamDetect to discard a message to this list.
Is this an ongoing issue? If not, perhaps the list's dmarc_moderation_action was changed subsequent to the last occurrence.
If this is a recurring issue, perhaps your SpamDetect.py handler has been modified in some way from the distribution one at https://bazaar.launchpad.net/~mailman-coders/mailman/2.1/view/head:/Mailman/.... What Mailman version is this? Installed how?
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (4)
-
Johannes Rohr -
jorohr＠gmail.com
-
Mark Sapiro -
Richard Damon