Charter occasionally bouncing mail
Hi,
One of my Mailman lists has a single member at Charter which has occasionally bounced mail over the last few days. When this happens, the reason given, when I look it up on their help page, indicates the message I sent goes against the security policies of my domain, and I should contact my domain administrator (that would be me). I have SPF and DKIM set up, and a quick check at dkimvalidator.com verifies they're both working. I assume this is one of these annoying situations where Charter is seeing what's clearly a transient DNS problem and treating it like a permanent failure? Also I assume there's nothing I can do about this? Is the problem likely to be at Charter's end or at my domain's nameservers' end?
Thanks,
Jayson
On 11/28/21 7:58 PM, Jayson Smith wrote:
Hi,
One of my Mailman lists has a single member at Charter which has occasionally bounced mail over the last few days. When this happens, the reason given, when I look it up on their help page, indicates the message I sent goes against the security policies of my domain, and I should contact my domain administrator (that would be me). I have SPF and DKIM set up, and a quick check at dkimvalidator.com verifies they're both working. I assume this is one of these annoying situations where Charter is seeing what's clearly a transient DNS problem and treating it like a permanent failure? Also I assume there's nothing I can do about this? Is the problem likely to be at Charter's end or at my domain's nameservers' end?
Only guessing, but this sounds like DMARC. Does your list apply DMARC mitigations?
If it is DMARC, the issue is the message sent to the charter subscriber is From: poster@posters.domain. posters.domain publishes a DMARC policy of (probably) reject. Yahoo.com is one such common domain. Your list modifies the message by content filtering, subject prefixing, adding msg_footer or some other transformation that breaks the posters.domain DKIM signature. Your SPF and DKIM signatures pass, but they are not 'aligned' with posters.domain, so they don't count for DMARC.
See https://wiki.list.org/DEV/DMARC
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi again,
Good point about DMARC. Does anyone know if Charter suddenly started caring about some DMARC policies on or around this past Friday? I have my list set to munge the From: lines of messages from senders E.G. AOL, Yahoo, etc. that publish a DMARC rejection policy.
On a slightly different topic, I've heard from a few Outlook users that list messages are consistently ending up in their junkmail folders. Could this be because Microsoft doesn't like the fact that my list is causing DMARC to fail, but not actually complaining to me about it? I could solve this problem by having the list munge the From: line for all messages, but sometimes that causes problems with replying. In particular, several years ago when my lists were set up to do that, Thunderbird users were having problems sometimes replying to the sender of a message rather than the entire list.
Jayson
On 11/28/2021 11:45 PM, Mark Sapiro wrote:
On 11/28/21 7:58 PM, Jayson Smith wrote:
Hi,
One of my Mailman lists has a single member at Charter which has occasionally bounced mail over the last few days. When this happens, the reason given, when I look it up on their help page, indicates the message I sent goes against the security policies of my domain, and I should contact my domain administrator (that would be me). I have SPF and DKIM set up, and a quick check at dkimvalidator.com verifies they're both working. I assume this is one of these annoying situations where Charter is seeing what's clearly a transient DNS problem and treating it like a permanent failure? Also I assume there's nothing I can do about this? Is the problem likely to be at Charter's end or at my domain's nameservers' end?
Only guessing, but this sounds like DMARC. Does your list apply DMARC mitigations?
If it is DMARC, the issue is the message sent to the charter subscriber is From: poster@posters.domain. posters.domain publishes a DMARC policy of (probably) reject. Yahoo.com is one such common domain. Your list modifies the message by content filtering, subject prefixing, adding msg_footer or some other transformation that breaks the posters.domain DKIM signature. Your SPF and DKIM signatures pass, but they are not 'aligned' with posters.domain, so they don't count for DMARC.
I have had a lot of experience with these things. Here are some observations. I have a list of 4000+ subscribers around the world. I have SPF and DKIM but not DMARC. (I never say much point in DMARC, and it does not seem necessary.) Right now every single one of the 4000+ subscribers accepts the mail, most of the time. Occasionally I get msssages (from Europe) saying that the mail has been blocked because it is a "high probability of spam" or "looks like spam". This drives me crazy. These spam-blocking systems are unregulated. They are like snake oil. They should not be blocking mail without telling the recipients, and this is what happens.
A few times, Microsoft has started blocking mail to ALL addresses with domains of outlook, hotmail, msn, or live. Sometimes this was the result of what you are talking about. I was told to sign up for various things, including "sender support": https://sendersupport.olc.protection.outlook.com/snds/ You can get data on what proportion of your mail counts as spam (if you have enough mail, as we do). When they block mail, you can complain: https://support.microsoft.com/supportrequestform/8ad563e3-288e-2a61-8122-3ba... (the one that works for me) or just https://support.microsoft.com/supportrequestform/
If you complain, you will get an automatic reply saying that your problem does not qualify for mitigation and that they are almost always correct. Then you have to respond to that. After a few rounds of this, you will get a response from what seems to be a human being, who will tell you that they are taking your problem very seriously, yada yada.
The last time this happened, they were completely blocking all make for over a week, because my IPV4 address (the one they use) was part of a range of addresses from which spam was being sent. Of course, I have only one ipv4 address (from a cloud server, Linode). The problem seems fixed for now, but I am warning new subscribers not to use Microsoft-controlled addresses.
Of course they won't tell you HOW they decide that something is spam, as this information would just make it easier for spammers.
(But I don't see what is so bad about spam. You just delete it; it helps if possible spam goes to a spefific folder, but any system I've seen makes many mistakes both ways, except spamassassin, which rarely makes a false positive. The real problem is phishing, and there have been no randomized control trials to see whether any system can immunize people against that. I doubt that these spam detectors do it effectively.)
Some references:
https://answers.microsoft.com/en-us/outlook_com/forum/all/hotmailoutlook-blo...
https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/C...
And there are several things like this: https://mxtoolbox.com/blacklists.aspx
But the list called UCEPROTECT3 (I think) is now, happily, widely ignored, because it is based on spam coming from a large range of ipv6 addresses on a cloud server. Spamhause does something like this too, but you can fix it by getting a "proper" ipv6 address that specifies the range ("/64" at the end).
Some geneneral
On 11/29/21 00:51, Jayson Smith wrote:
Hi again,
Good point about DMARC. Does anyone know if Charter suddenly started caring about some DMARC policies on or around this past Friday? I have my list set to munge the From: lines of messages from senders E.G. AOL, Yahoo, etc. that publish a DMARC rejection policy.
On a slightly different topic, I've heard from a few Outlook users that list messages are consistently ending up in their junkmail folders. Could this be because Microsoft doesn't like the fact that my list is causing DMARC to fail, but not actually complaining to me about it? I could solve this problem by having the list munge the From: line for all messages, but sometimes that causes problems with replying. In particular, several years ago when my lists were set up to do that, Thunderbird users were having problems sometimes replying to the sender of a message rather than the entire list.
Jayson
On 11/28/2021 11:45 PM, Mark Sapiro wrote:
On 11/28/21 7:58 PM, Jayson Smith wrote:
Hi,
One of my Mailman lists has a single member at Charter which has occasionally bounced mail over the last few days. When this happens, the reason given, when I look it up on their help page, indicates the message I sent goes against the security policies of my domain, and I should contact my domain administrator (that would be me). I have SPF and DKIM set up, and a quick check at dkimvalidator.com verifies they're both working. I assume this is one of these annoying situations where Charter is seeing what's clearly a transient DNS problem and treating it like a permanent failure? Also I assume there's nothing I can do about this? Is the problem likely to be at Charter's end or at my domain's nameservers' end?
Only guessing, but this sounds like DMARC. Does your list apply DMARC mitigations?
If it is DMARC, the issue is the message sent to the charter subscriber is From: poster@posters.domain. posters.domain publishes a DMARC policy of (probably) reject. Yahoo.com is one such common domain. Your list modifies the message by content filtering, subject prefixing, adding msg_footer or some other transformation that breaks the posters.domain DKIM signature. Your SPF and DKIM signatures pass, but they are not 'aligned' with posters.domain, so they don't count for DMARC.
Mailman-Users mailing list -- mailman-users@python.org To unsubscribe send an email to mailman-users-leave@python.org https://mail.python.org/mailman3/lists/mailman-users.python.org/ Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/ https://mail.python.org/archives/list/mailman-users@python.org/
-- Jonathan Baron, Professor of Psychology, University of Pennsylvania Home page: https://www.sas.upenn.edu/~baron Founding Editor: Judgment and Decision Making (http://journal.sjdm.org)
Hi,
Don't talk to me about the Microsoft blocklist! I've just had a recent experience of same.
On June 11, 2015, my IP was blocked. No mail was accepted by their servers. You know the drill: "Unfortunately, messages from [XX>.XX.XX.XX] weren't sent. Please contact your Internet service provider since part of their network is on our block list (S3150). You can also refer your provider to http://mail.live.com/mail/troubleshooting.aspx#errors. [AM5EUR02FT013.eop-EUR02.prod.protection.outlook.com]" This was my first experience with their blocklist or, for that matter, any ISP internal blocklist. I signed up for their junkmail reporting program and smart network data services. The next morning I figured out how to properly submit a deliverability support ticket. About half an hour later, they responded, indicating my IP was conditionally mitigated. End of story…until September 26, 2018. Same thing all over again. Sent a deliverability support request, got my IP mitigated. Keep in mind that during both these incidents I was running a very high traffic mailing list. After the second incident, I decided to move this particular list to groups.io so that traffic would no longer be coming from my IP address. That move went off without a hitch, and all my problems with the Microsoft internal blocklist were behind me…
Please place your cassette player in fast forward mode until you reach last Saturday, November 20, 2021. I woke up in the evening to some bounce reports. You guessed it, Microsoft's blocklist reared its ugly head again. Oh well, no big deal, I'll just fill out a deliverability support request. For some reason it took several hours for them to confirm my ticket. Okay this is good, I should be good to go in a bit…not so much, as it turns out. When the response came in, I was shocked to learn that my IP was "not qualified for mitigation." Not qualified for mitigation? What? This had me extremely worried that maybe someone had hacked my server and was sending out spam of which I was unaware or something. So in a panic, I composed a letter detailing what types of mail I send from my IP. Sent the message. Get a reply that they're looking into it. Several anxious hours go by, in which I'm worrying myself to death that they're going to reply and say, "We're not going to mitigate anything for you, and we're not going to tell you why, and there's nothing you can do about it." Then I finally! get the response I've been waiting for, indicating that they've implemented mitigation for my IP. Since they mitigated the problem, I assume that means, in reality, I wasn't really doing anything that horrible in the first place or they wouldn't have mitigated, but that makes me wonder why the clearly automated "investigation" mitigated my IP twice, then wouldn't the third time? Maybe it assumes if you get on their blocklist three times there's clearly something wrong, even if each incident is years apart? Who knows?
Jayson
On 11/29/2021 6:55 AM, Jon Baron wrote:
I have had a lot of experience with these things. Here are some observations. I have a list of 4000+ subscribers around the world. I have SPF and DKIM but not DMARC. (I never say much point in DMARC, and it does not seem necessary.) Right now every single one of the 4000+ subscribers accepts the mail, most of the time. Occasionally I get msssages (from Europe) saying that the mail has been blocked because it is a "high probability of spam" or "looks like spam". This drives me crazy. These spam-blocking systems are unregulated. They are like snake oil. They should not be blocking mail without telling the recipients, and this is what happens.
A few times, Microsoft has started blocking mail to ALL addresses with domains of outlook, hotmail, msn, or live. Sometimes this was the result of what you are talking about. I was told to sign up for various things, including "sender support": https://sendersupport.olc.protection.outlook.com/snds/ You can get data on what proportion of your mail counts as spam (if you have enough mail, as we do). When they block mail, you can complain: https://support.microsoft.com/supportrequestform/8ad563e3-288e-2a61-8122-3ba... (the one that works for me) or just https://support.microsoft.com/supportrequestform/
If you complain, you will get an automatic reply saying that your problem does not qualify for mitigation and that they are almost always correct. Then you have to respond to that. After a few rounds of this, you will get a response from what seems to be a human being, who will tell you that they are taking your problem very seriously, yada yada.
The last time this happened, they were completely blocking all make for over a week, because my IPV4 address (the one they use) was part of a range of addresses from which spam was being sent. Of course, I have only one ipv4 address (from a cloud server, Linode). The problem seems fixed for now, but I am warning new subscribers not to use Microsoft-controlled addresses.
Of course they won't tell you HOW they decide that something is spam, as this information would just make it easier for spammers.
(But I don't see what is so bad about spam. You just delete it; it helps if possible spam goes to a spefific folder, but any system I've seen makes many mistakes both ways, except spamassassin, which rarely makes a false positive. The real problem is phishing, and there have been no randomized control trials to see whether any system can immunize people against that. I doubt that these spam detectors do it effectively.)
Some references:
https://answers.microsoft.com/en-us/outlook_com/forum/all/hotmailoutlook-blo...
https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/C...
And there are several things like this: https://mxtoolbox.com/blacklists.aspx
But the list called UCEPROTECT3 (I think) is now, happily, widely ignored, because it is based on spam coming from a large range of ipv6 addresses on a cloud server. Spamhause does something like this too, but you can fix it by getting a "proper" ipv6 address that specifies the range ("/64" at the end).
Some geneneral
On 11/29/21 00:51, Jayson Smith wrote:
Hi again,
Good point about DMARC. Does anyone know if Charter suddenly started caring about some DMARC policies on or around this past Friday? I have my list set to munge the From: lines of messages from senders E.G. AOL, Yahoo, etc. that publish a DMARC rejection policy.
On a slightly different topic, I've heard from a few Outlook users that list messages are consistently ending up in their junkmail folders. Could this be because Microsoft doesn't like the fact that my list is causing DMARC to fail, but not actually complaining to me about it? I could solve this problem by having the list munge the From: line for all messages, but sometimes that causes problems with replying. In particular, several years ago when my lists were set up to do that, Thunderbird users were having problems sometimes replying to the sender of a message rather than the entire list.
Jayson
On 11/28/2021 11:45 PM, Mark Sapiro wrote:
On 11/28/21 7:58 PM, Jayson Smith wrote:
Hi,
One of my Mailman lists has a single member at Charter which has occasionally bounced mail over the last few days. When this happens, the reason given, when I look it up on their help page, indicates the message I sent goes against the security policies of my domain, and I should contact my domain administrator (that would be me). I have SPF and DKIM set up, and a quick check at dkimvalidator.com verifies they're both working. I assume this is one of these annoying situations where Charter is seeing what's clearly a transient DNS problem and treating it like a permanent failure? Also I assume there's nothing I can do about this? Is the problem likely to be at Charter's end or at my domain's nameservers' end?
Only guessing, but this sounds like DMARC. Does your list apply DMARC mitigations?
If it is DMARC, the issue is the message sent to the charter subscriber is From: poster@posters.domain. posters.domain publishes a DMARC policy of (probably) reject. Yahoo.com is one such common domain. Your list modifies the message by content filtering, subject prefixing, adding msg_footer or some other transformation that breaks the posters.domain DKIM signature. Your SPF and DKIM signatures pass, but they are not 'aligned' with posters.domain, so they don't count for DMARC.
Mailman-Users mailing list -- mailman-users@python.org To unsubscribe send an email to mailman-users-leave@python.org https://mail.python.org/mailman3/lists/mailman-users.python.org/ Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/ https://mail.python.org/archives/list/mailman-users@python.org/
Jayson Smith writes:
Good point about DMARC. Does anyone know if Charter suddenly started caring about some DMARC policies on or around this past Friday?
I for one don't know. You'd have to ask their postmaster, or get the subscriber to do so, to be sure. I think it's as likely that they got an update to their filters from a vendor. That could be sensitive to DMARC from alignment, or it could be something else.
I have my list set to munge the From: lines of messages from senders E.G. AOL, Yahoo, etc. that publish a DMARC rejection policy.
You could try setting up your list mail to participate in the ARC protocol.[1] I think most MTAs have options or plugins for this by now. Also Mailman 3 has an option to handle it itself, but it is preferable for the MTA to handle it as Mailman 3 can't validate SPF.
On a slightly different topic, I've heard from a few Outlook users that list messages are consistently ending up in their junkmail folders.
All of the big providers have this problem occasionally, although my impression that it's more of a problem with Microsoft than Google or Yahoo!. Again, ARC might help.
Footnotes: [1] Authenticated Received Chain, https://datatracker.ietf.org/doc/html/rfc8617
On Nov 29, 2021, at 5:31 AM, Stephen J. Turnbull <stephenjturnbull@gmail.com<mailto:stephenjturnbull@gmail.com>> wrote:
Jayson Smith writes:
Good point about DMARC. Does anyone know if Charter suddenly started caring about some DMARC policies on or around this past Friday?
I for one don't know. You'd have to ask their postmaster, or get the subscriber to do so, to be sure. I think it's as likely that they got an update to their filters from a vendor. That could be sensitive to DMARC from alignment, or it could be something else.
I have my list set to munge the From: lines of messages from senders E.G. AOL, Yahoo, etc. that publish a DMARC rejection policy.
You could try setting up your list mail to participate in the ARC protocol.[1] I think most MTAs have options or plugins for this by now. Also Mailman 3 has an option to handle it itself, but it is preferable for the MTA to handle it as Mailman 3 can't validate SPF.
On a slightly different topic, I've heard from a few Outlook users that list messages are consistently ending up in their junkmail folders.
All of the big providers have this problem occasionally, although my impression that it's more of a problem with Microsoft than Google or Yahoo!. Again, ARC might help.
Yes, this mailman list has suddenly started going into my junk folder.
If you can get the problematic user to forward the entire message (forward as attachment) you can examine the headers for the source that sent it to junk mail; normally there are several. They will usualy have some simple-to-cryptic message in the headers that will say how it examined the message, what score it gave it and what score is required. There are also headers for DKIM status and SPF status.
For example, I think this is the one from our Barracuda system (I think) :
X-BESS-Spam-Status: SCORE=0.50 using account:ESS35309 scores of QUARANTINE_LEVEL=0.0 KILL_LEVEL=5.0 tests=BSF_SC0_SA085, BSF_SC0_SA085b, BSF_SC0_MISMATCH_TO Received-SPF: pass (mx-inbound46-171.us<http://mx-inbound46-171.us>-east-2c.ess.aws.cudaops.com: domain of mailman-users-bounces+johnson=pharmacy.arizona.edu@python.org<mailto:mailman-users-bounces+johnson=pharmacy.arizona.edu@python.org> designates 188.166.95.178 as permitted sender) X-BESS-Spam-Report: Code version 3.2, rules version 3.2.2.236178 [from cloudscan13- 165.us<http://165.us>-east-2a.ess.aws.cudaops.com] Rule breakdown below pts rule name description
0.10 BSF_SC0_SA085 META: Custom Rule SA085 0.40 BSF_SC0_SA085b META: Custom Rule SA085b 0.00 BSF_SC0_MISMATCH_TO META: Envelope rcpt doesn't match header X-BESS-Spam-Score: 0.50 X-BESS-BRTS-Status: 1
So this is the last step before O365, and in my case it’s %@$#*#& Microsoft O365, because we have a hybrid Exchange setup and our email is delivered through our Barracuda antispam servers, so the O365 anti-spam process is starved of spam, and apparently has quotas to meet or it’ll be fired, so it just false-positives a ton of messages, and sadly they’re mainly Mailman lists I NEED to keep up on, and their whitelisting process is, inexplicably, entirely based on the From: sender address. :-(
X-Microsoft-Antispam-Mailbox-Delivery: ucf:0;jmr:0;auth:0;dest:J;OFR:SpamFilterAuthJ;ENG:(910001)(944506458)(944626604)(920097)(930097)(3100021);RF:JunkEmail;
This has just started happening in the last few weeks, so I think they’ve turned on some stupid setting; my Exchange365 settings are to NOT send anything to my junk folder.
I've had issues with other mailing lists, and in one we have to munge the ‘from’ addresses for all traffic not just the Yahoo ones :-/
Is this a user-enableable setting?
-- Bruce Johnson University of Arizona College of Pharmacy Information Technology Group
Institutions do not have opinions, merely customs
On 11/28/21 9:51 PM, Jayson Smith wrote:
Hi again,
Good point about DMARC. Does anyone know if Charter suddenly started caring about some DMARC policies on or around this past Friday? I have my list set to munge the From: lines of messages from senders E.G. AOL, Yahoo, etc. that publish a DMARC rejection policy.
If your list has dmarc_moderation_action set to Munge From, that should avoid DMARC issues. If it does not also have dmarc_quarantine_moderation_action set to Yes, setting that may help.
On a slightly different topic, I've heard from a few Outlook users that list messages are consistently ending up in their junkmail folders. Could this be because Microsoft doesn't like the fact that my list is causing DMARC to fail, but not actually complaining to me about it?
If you don't have dmarc_quarantine_moderation_action set to Yes, that could be the reason.
I could solve this problem by having the list munge the From: line for all messages, but sometimes that causes problems with replying. In particular, several years ago when my lists were set up to do that, Thunderbird users were having problems sometimes replying to the sender of a message rather than the entire list.
We do our best with placing the original From: in Reply-To: or Cc: to make 'reply' and 'reply-all' behavior consistent between munged and non-munged messages, but ultimately this depends on the MUA doing the replying.
That said, with Munge From applied to messages From: domains publishing DMARC reject or quarantine, applying it to all messages shouldn't change this.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (5)
-
Bruce Johnson -
Jayson Smith -
Jon Baron -
Mark Sapiro -
Stephen J. Turnbull