Hello!
I believe there's no need to elaborate on the problems recently introduced by Yahoo, changing their DMARC DNS record and rendering many mailman lists unusable for Yahoo mail users.
I see the solution to this problem in changing the From: field to mailing list's address, but keeping the poster's name or address in the description part of the same field. For example:
From: "List Name on behalf of Poster Name" <list@add.re.ss>or From: "Poster Name via List Name" <list@add.re.ss> or From: "Poster Name" <list@add.re.ss> or From: "Poster Name [poster@add.re.ss]" <list@add.re.ss>
I'm using Mailman 2.1.13, and can not upgrade to 2.1.16 on a live system. I was forced to turn on anonymous_list as the urgent remedy, byt the full anonymization is not really how it should be.
Could someone please help me achieve this using the above version, by some changes in the code?
Thank you!
-- Pozdrav / Regards, Siniša Burina
They're breaking RFC 822 / 5322.
The "From:" field specifies the author(s) of the message, that is, the mailbox(es) of the person(s) or system(s) responsible for the writing of the message. [...] In all cases, the "From:" field SHOULD NOT contain any mailbox that does not belong to the author(s) of the message.
I don't think we should compound that by changing the From line.
Joseph Brennan Columbia University Information Technology
On 10/04/14 16:25, Joseph Brennan wrote:
They're breaking RFC 822 / 5322.
The "From:" field specifies the author(s) of the message, that is, the mailbox(es) of the person(s) or system(s) responsible for the writing of the message. [...] In all cases, the "From:" field SHOULD NOT contain any mailbox that does not belong to the author(s) of the message.
I don't think we should compound that by changing the From line.
Well, anonymous_list option does that too, completely hiding the original poster's information. The approach I proposed would do the same, only in slightly subtler manner. :)
-- Pozdrav / Regards, Siniša Burina
I hate to say it, but the days of the kinder, gentler internet when everyone played strictly by the RFCs are passing as operational control of internet services comes increasingly under the control of fewer, bigger players who can do as they wish.
This isn't to say that Mailman should break RFCs too, but there are few options. SPF inherently breaks mailing lists, which is why heretofore it's been used mostly as an advisory protocol rather than as a determinant of whether an email gets delivered, or not. I understand that SPF is one of the components of DMARC protocols.
This is the first I've heard of this issue, but it doesn't surprise me at all.
On Thu, 2014-04-10 at 10:25 -0400, Joseph Brennan wrote:
They're breaking RFC 822 / 5322.
The "From:" field specifies the author(s) of the message, that is, the mailbox(es) of the person(s) or system(s) responsible for the writing of the message. [...] In all cases, the "From:" field SHOULD NOT contain any mailbox that does not belong to the author(s) of the message.
I don't think we should compound that by changing the From line.
Joseph Brennan Columbia University Information Technology
Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/fmouse%40fmp.com
-- Lindsay Haisley | "Everything works if you let it" FMP Computer Services | 512-259-1190 | --- The Roadie http://www.fmp.com |
Lindsay Haisley <fmouse@fmp.com> wrote:
SPF inherently breaks mailing lists
No, it doesn't. SPF checks the envelope sender, and when the list host is, say, lists.example.com, the envelope sender is something like listname-bounces@lists.example.com, and that can pass SPF. Mailman, Listserv, etc, all write their envelope sender that way.
Joseph Brennan Columbia University Information Technology
On 10/04/14 17:18, Lindsay Haisley wrote:
This is the first I've heard of this issue, but it doesn't surprise me at all.
Basically, Yahoo insists that their own mail servers are the only ones that can originate the message with @yahoo.com domain in the From header. Not Return-Path, Not the envelope sender, but exactly the From header in the message itself. If this practice gets adopted by more organizations, I don't know how else could this problem be solved.
-- Pozdrav / Regards, Siniša Burina
Siniša Burina writes:
Basically, Yahoo insists that their own mail servers are the only ones that can originate the message with @yahoo.com domain in the From header. Not Return-Path, Not the envelope sender, but exactly the From header in the message itself. If this practice gets adopted by more organizations, I don't know how else could this problem be solved.
If yahoo wants to give their users an excuse to use a different address and stop advertising yahoo, I have no problem with that. :-)
The straightforward thing for Mailman to do is to wrap mail from yahoo addresses in a multipart/mixed with a text part explaining that Yahoo is "knowingly interfering with" the mail service of their users, and the mail itself in a message/rfc822 part. As far as I know, no component of DMARC allows digging into a message and trying to DMARC the MIME parts.
Or just bounce them with a message stating that Yahoo no longer permits its users to post to mailing lists, so please use a different posting address. I realize that most sites can't do that, but mine can (and will if I get any complaints about this policy -- my subscribers are sympathetic).
Steve
On 10/04/14 19:57, Stephen J. Turnbull wrote:
Or just bounce them with a message stating that Yahoo no longer permits its users to post to mailing lists, so please use a different posting address. I realize that most sites can't do that, but mine can (and will if I get any complaints about this policy -- my subscribers are sympathetic).
And that's exactly what I'm going to do. :)
-- Pozdrav / Regards, Siniša Burina
On Thu, Apr 10, 2014 at 2:34 PM, Siniša Burina <six@burina.net> wrote:
On 10/04/14 19:57, Stephen J. Turnbull wrote:
Or just bounce them with a message stating that Yahoo no longer permits its users to post to mailing lists, so please use a different posting address. I realize that most sites can't do that, but mine can (and will if I get any complaints about this policy -- my subscribers are sympathetic).
And that's exactly what I'm going to do. :)
Here's a tried and tested patch just awaiting more use:
https://code.launchpad.net/~jimpop/mailman/dmarc-reject
-Jim P.
On Thu, 2014-04-10 at 15:35 -0400, Jim Popovitch wrote:
Here's a tried and tested patch just awaiting more use:
Jim, I note that what you reference here appears to be a complete branch version of Mailman. Can you briefly outline exactly what mods you've made and how they deal with the DMARC problem, as well as any other differences between your patch and the main Mailman trunk? Or refer me to a description of this within the file set. I can read the code, but maybe you could save me a bit of time.
I have only a few Mailman discussion lists at FMP, most of them pro bono. After considering caving in on this and breaking the RFC regarding the proper use of the From header, I'm inclinded to favor Stephen's suggestion of bouncing messages from mail service providers enforcing DMARC rejection criteria. Does you patch set do this?
-- Lindsay Haisley | "Everything works if you let it" FMP Computer Services | 512-259-1190 | --- The Roadie http://www.fmp.com |
On Sun, Apr 13, 2014 at 8:25 PM, Lindsay Haisley <fmouse@fmp.com> wrote:
On Thu, 2014-04-10 at 15:35 -0400, Jim Popovitch wrote:
Here's a tried and tested patch just awaiting more use:
Jim, I note that what you reference here appears to be a complete branch version of Mailman. Can you briefly outline exactly what mods you've made and how they deal with the DMARC problem, as well as any other differences between your patch and the main Mailman trunk? Or refer me to a description of this within the file set. I can read the code, but maybe you could save me a bit of time.
Hi Lindsey,
Launchpad is not my forte, so I'm not even sure how to push only my modifications to LP without the upstream branch changes. And yes, I'll be the first to say it's a bit confusing in it's current branch.
Here's a link to review the changes between rev 1375 and 1380 (where my additions were added).
http://bazaar.launchpad.net/~jimpop/mailman/dmarc-reject/revision/1375?remem...
I have only a few Mailman discussion lists at FMP, most of them pro bono. After considering caving in on this and breaking the RFC regarding the proper use of the From header, I'm inclinded to favor Stephen's suggestion of bouncing messages from mail service providers enforcing DMARC rejection criteria. Does you patch set do this?
Yes, specifically that. It checks for a DMARC domain record on every list post. The patch allows you do define, per list, whether to allow/disgard/reject/hold if a poster's domain has a DMARC p=reject policy published.
-Jim P.
On Sun, Apr 13, 2014 at 8:48 PM, Jim Popovitch <jimpop@gmail.com> wrote:
On Sun, Apr 13, 2014 at 8:25 PM, Lindsay Haisley <fmouse@fmp.com> wrote:
On Thu, 2014-04-10 at 15:35 -0400, Jim Popovitch wrote:
Here's a tried and tested patch just awaiting more use:
Jim, I note that what you reference here appears to be a complete branch version of Mailman. Can you briefly outline exactly what mods you've made and how they deal with the DMARC problem, as well as any other differences between your patch and the main Mailman trunk? Or refer me to a description of this within the file set. I can read the code, but maybe you could save me a bit of time.
Hi Lindsey,
Launchpad is not my forte, so I'm not even sure how to push only my modifications to LP without the upstream branch changes. And yes, I'll be the first to say it's a bit confusing in it's current branch.
Here's a link to review the changes between rev 1375 and 1380 (where my additions were added).
http://bazaar.launchpad.net/~jimpop/mailman/dmarc-reject/revision/1375?remem...
It's also worth mentioning that this patch needs Python's dns.resolver (libnet-dns-perl on Debian) to function successfully. This reminds me that I need to add a README for this code, which I will do sometime in the next day or two. If you have any questions, feel free to ask.
-Jim P.
On Apr 13, 2014, at 08:48 PM, Jim Popovitch wrote:
Launchpad is not my forte, so I'm not even sure how to push only my modifications to LP without the upstream branch changes. And yes, I'll be the first to say it's a bit confusing in it's current branch.
The best way to do it is to submit a merge proposal against the lp:mailman/2.1 branch (it would be against lp:mailman if it were for 3.0). Then Launchpad will create an easily displayable diff, e.g.
https://code.launchpad.net/~jimpop/mailman/dmarc-reject/+merge/215591
Cheers, -Barry
On Sun, Apr 13, 2014 at 9:06 PM, Barry Warsaw <barry@list.org> wrote:
On Apr 13, 2014, at 08:48 PM, Jim Popovitch wrote:
Launchpad is not my forte, so I'm not even sure how to push only my modifications to LP without the upstream branch changes. And yes, I'll be the first to say it's a bit confusing in it's current branch.
The best way to do it is to submit a merge proposal against the lp:mailman/2.1 branch (it would be against lp:mailman if it were for 3.0). Then Launchpad will create an easily displayable diff, e.g.
https://code.launchpad.net/~jimpop/mailman/dmarc-reject/+merge/215591
Ahh, Thank you Barry, and I'll take that 3.0 hint and begin working on that patch. ;-)
-Jim P.
On Apr 11, 2014, at 02:57 AM, Stephen J. Turnbull wrote:
The straightforward thing for Mailman to do is to wrap mail from yahoo addresses in a multipart/mixed with a text part explaining that Yahoo is "knowingly interfering with" the mail service of their users, and the mail itself in a message/rfc822 part. As far as I know, no component of DMARC allows digging into a message and trying to DMARC the MIME parts.
That's what I mean by "we can fix it if we make the user experience horrible". See all the complaints about the MIME-proper way we add footers in some cases.
-Barry
On Thu, Apr 10, 2014 at 10:18:33AM -0500, Lindsay Haisley wrote:
I hate to say it, but the days of the kinder, gentler internet when everyone played strictly by the RFCs are passing as operational control of internet services comes increasingly under the control of fewer, bigger players who can do as they wish.
+1.
--
If all else fails, immortality can always
be assured by spectacular error.
-- Galbraith
On Thu, Apr 10, 2014 at 12:04 PM, Adam McGreggor <adam-mailman@amyl.org.uk> wrote:
On Thu, Apr 10, 2014 at 10:18:33AM -0500, Lindsay Haisley wrote:
I hate to say it, but the days of the kinder, gentler internet when everyone played strictly by the RFCs are passing as operational control of internet services comes increasingly under the control of fewer, bigger players who can do as they wish.
+1.
Further to that point: The behemoths doing this also offer competitive (revenue based!) offerings to the services they are impacting.
-Jim P.
On 04/10/2014 07:25 AM, Joseph Brennan wrote:
They're breaking RFC 822 / 5322.
The "From:" field specifies the author(s) of the message, that is, the mailbox(es) of the person(s) or system(s) responsible for the writing of the message. [...] In all cases, the "From:" field SHOULD NOT contain any mailbox that does not belong to the author(s) of the message.
I don't think we should compound that by changing the From line.
Several others have made relevant replies while I was on the plane to Montreal, registering for PyCon, attending the opening reception ...
Anyway, I wanted to say I agree completely with the above even though the DMARC community does not. That's why I implemented the option in 2.1.16 to wrap the original post as a message/rfc822 part attached to a new message from the list.
Unfortunately, when I actually turned this on in response to Yahoo's change in DMARC policy, I got complaints from users of Apple iOS iThings that their mail clients do not deal well with this message, so I reluctantly went the non-compliant mung the From: way.
Yesterday I wrote a brief FAQ on this which is at <http://wiki.list.org/x/ggARAQ>.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Apr 10, 2014, at 20:25, Mark Sapiro <mark@msapiro.net> wrote:
On 04/10/2014 07:25 AM, Joseph Brennan wrote:
They're breaking RFC 822 / 5322.
The "From:" field specifies the author(s) of the message, that is, the mailbox(es) of the person(s) or system(s) responsible for the writing of the message. [...] In all cases, the "From:" field SHOULD NOT contain any mailbox that does not belong to the author(s) of the message.
I don't think we should compound that by changing the From line.
Several others have made relevant replies while I was on the plane to Montreal, registering for PyCon, attending the opening reception ...
Anyway, I wanted to say I agree completely with the above even though the DMARC community does not. That's why I implemented the option in 2.1.16 to wrap the original post as a message/rfc822 part attached to a new message from the list.
Unfortunately, when I actually turned this on in response to Yahoo's change in DMARC policy, I got complaints from users of Apple iOS iThings that their mail clients do not deal well with this message, so I reluctantly went the non-compliant mung the From: way.
Yesterday I wrote a brief FAQ on this which is at <http://wiki.list.org/x/ggARAQ>.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
At least as of iOS 7 it can show messages inside messages.
That's how I view email forwarded from Exchange.
Now if only bottom posting was easier on an iPhone.
Fat fingered from my iPhone -- miscorrections happen.
On 04/10/2014 05:35 PM, mail.ulticom.com wrote:
At least as of iOS 7 it can show messages inside messages.
Thanks for the tip. I'll check with my users and see what they're using.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro writes:
Unfortunately, when I actually turned this on in response to Yahoo's change in DMARC policy, I got complaints from users of Apple iOS iThings that their mail clients do not deal well with this message,
The iOS 6 mail client was just plain unusable, and in very limited experience the iOS 7.1 client is not a lot better. Neither can stay synched with the state of Gmail on my PC. It's easier to use Gmail from Safari (even though Safari has trouble displaying those pages correctly), and I'm going to try the Gmail iPhone client later today (I normally don't process mail from my iPhone 4S/iOS 7.1).
I can't speak for those who don't use Gmail, of course, but I find it hard to be sympathetic with people who complain that the very limited clients provided by Apple are, well, *very* limited.
so I reluctantly went the non-compliant mung the From: way.
That's a shame. I really think putting the blame on Yahoo! and the DMARC advocates (Yahoo! clearly being a leader in that crowd), where it belongs, and the discomfort on Yahoo! users, is a better idea.
On Apr 10, 2014, at 03:09 PM, Siniša Burina wrote:
I believe there's no need to elaborate on the problems recently introduced by Yahoo, changing their DMARC DNS record and rendering many mailman lists unusable for Yahoo mail users.
It *is* a shame that these anti-spam defenses knowingly break mailing lists. I say "knowingly" but not "maliciously" because their specs usually describe the adverse affects on mailing lists, along with some mitigation approaches (which may not have a positive effect on list usability <wink>). The spec authors are not hostile to mailing lists and would rather not break them, but there does seem to be a fundamental conflict between mailing lists and anti-spam approaches.
That said, DMARC was discussed in great detail last year on the -developers list, so if you want all the gory details, check out those archives.
Mark will probably follow up in more detail, but MM2.1 implemented a feature in 2.1.16 called "from_is_list" which is a ternary option for addressing the effects of DMARC. It has to be enabled by the site admin, and then list admins can opt-in. It's disabled by default for backward compatibility reasons. From Defaults.py:
# The following is a three way setting. # 0 -> Do not rewrite the From: or wrap the message. # 1 -> Rewrite the From: header of posts replacing the posters address with # that of the list. Also see REMOVE_DKIM_HEADERS above. # 2 -> Do not modify the From: of the message, but wrap the message in an outer # message From the list address. DEFAULT_FROM_IS_LIST = 0
So as you can see, two approaches are available, From: rewriting or outer message wrapping. Both are suboptimal for usability, but it seems like we have no other viable option. This is not yet implemented in MM3 because I don't really like having to do it. We might have no choice though.
-Barry
(my apologies to anyone who reads NANOG, this is mostly a repeat of what I said there)
On Thu, Apr 10, 2014 at 11:36:16AM -0400, Barry Warsaw wrote:
It *is* a shame that these anti-spam defenses knowingly break mailing lists.
It's a shame that this is being pushed as an anti-spam defense when in fact (a) it has little-to-no anti-spam value and (b) measures that have much higher anti-spam value with few adverse effects are not being used.
Nearly all (at least 99% and likely quite a bit more) of the spam [as observed by my numerous spamtraps] that purports to originate from Yahoo really *does* originate from Yahoo. All that I have to do to verify that is to look at the originating host -- that is, it's not necessary to check DMARC or anything else.
There are several reasons for this. First, Yahoo has done an absolutely miserable job of outbound abuse control. For over a decade. Second, they've done a correspondingly miserable job of handling abuse reports, so even when one of their victims is kind and generous enough to do their work for them and tell them that they have a problem...they don't pay attention and they don't take any action. (Or they fire back a clueless boilerplate denial that it was their user on their host on their network...even though it was all three.) Also for over a decade. Third, why would any spammer forge a @yahoo.com address when it's easy enough to buy hijacked accounts by the bucketful -- or to use any of the usual exploits to go get some? Fourth, at least some spammers seem to have caught on that Yahoo isn't *worth* forging: it's a toxic cesspool because the people running it have allowed it to be become one.
So let's not pretend that this has anything to do with stopping spam. If Yahoo actually wanted to do something about spam, they could have done that years and years ago simply by *paying attention* to what was going on inside their own operation. This is just (a) propaganda, so that they claim to be "doing something" and (b) a clumsy attempt to coerce people into using *their* mailing lists, which are just as horribly run as the rest of their mail system.
---rsk
On 04/10/2014 06:09 AM, Siniša Burina wrote:
I see the solution to this problem in changing the From: field to mailing list's address, but keeping the poster's name or address in the description part of the same field. For example:
...
I'm using Mailman 2.1.13, and can not upgrade to 2.1.16 on a live system. I was forced to turn on anonymous_list as the urgent remedy, byt the full anonymization is not really how it should be.
Could someone please help me achieve this using the above version, by some changes in the code?
I'm not sure why you can't upgrade if you can patch the code, but in any case, I can't point you at a single patch to do it my way because there are several. You could do it by applying all of the following patches in order.
http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/diff/1402 http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/diff/1404 http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/diff/1415 http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/diff/1417 http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/diff/1418 http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/diff/1419 http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/diff/1446 http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/diff/1450 http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/diff/1453 http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/diff/1454 http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/diff/1455
(1415, 1417 and 1418 are i18n only)
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 11/04/14 03:19, Mark Sapiro wrote:
I'm not sure why you can't upgrade if you can patch the code, but in any case, I can't point you at a single patch to do it my way because there are several. You could do it by applying all of the following patches in order.
Thank you very much, Mark!
-- Pozdrav / Regards, Siniša Burina
I hadn't heard of this till now. Could somebody please confirm if my understanding of the issue is correct?
This is what I'm thinking will happen, please correct where I'm wrong:
- A list member sends an email to the list from a yahoo address
- The list sends that email out to all the list members
- The recipients' mail servers will (might?) check with yahoo what to do with the email, and will be advised to reject it
- The list will receive a bounce for every email address whose mail server follows that advice
- Those recipients whose mail server follows the advice will not receive the message
- The list will increment the bounce score for all those affected receipients, but only once per day
- The increment will be 1 because this is a hard bounce
- If the score reaches the bounce_score_threshold before the bounce_info_stale_after number of days has passed since the most recent bounce, then the member's subscription is disabled.
If that's correct then my understanding is that:
- If a list has at least one active yahoo member then pretty soon everyone's subscription will be disabled (not unsubscribed?).
- If a list receives vey few messages from yahoo addresses then the only effect will be that their messages don't get through, and that they might still get through to some people.
I'm a moderator for a cpanel list, but don't have access to any of the settings. Can someone tell me what the default settings are for bounce_score_threshold and bounce_info_stale_after? I'm assuming ours might still be whatever the defaults are.
Am I right in thinking that if we make these values high enough, we'll see no accounts disabled, and the only side effects will be more bounces and yahoo mail won't get through? Would this be an acceptable solution for a list with only 1000 members and low traffic, assuming we warn the yahoo members to use a different address?
Peter Shute
Siniša Burina wrote: I believe there's no need to elaborate on the problems recently introduced by Yahoo, changing their DMARC DNS record and rendering many mailman lists unusable for Yahoo mail users.
On 04/11/2014 06:28 PM, Peter Shute wrote:
I hadn't heard of this till now. Could somebody please confirm if my understanding of the issue is correct?
This is what I'm thinking will happen, please correct where I'm wrong:
- A list member sends an email to the list from a yahoo address
- The list sends that email out to all the list members
- The recipients' mail servers will (might?) check with yahoo what to do with the email, and will be advised to reject it
- The list will receive a bounce for every email address whose mail server follows that advice
- Those recipients whose mail server follows the advice will not receive the message
- The list will increment the bounce score for all those affected receipients, but only once per day
- The increment will be 1 because this is a hard bounce
- If the score reaches the bounce_score_threshold before the bounce_info_stale_after number of days has passed since the most recent bounce, then the member's subscription is disabled.
Correct.
If that's correct then my understanding is that:
- If a list has at least one active yahoo member then pretty soon everyone's subscription will be disabled (not unsubscribed?).
Everyone whose ISP honors Yahoo's DMARC reject policy. And they will eventually be unsubscribed after (bounce_you_are_disabled_warnings) * (bounce_you_are_disabled_warnings_interval) days.
- If a list receives vey few messages from yahoo addresses then the only effect will be that their messages don't get through, and that they might still get through to some people.
Maybe. Yahoo requests and receives reports of rejected mail. This is only speculation, but if Yahoo sees that your server is sending what it considers to be bogus mail purporting to be From: its domain, it could decide to reject all mail from your server.
I'm a moderator for a cpanel list, but don't have access to any of the settings. Can someone tell me what the default settings are for bounce_score_threshold and bounce_info_stale_after? I'm assuming ours might still be whatever the defaults are.
The list admin can see these values on the list's web admin Bounce processing page, but defaults are:
bounce_score_threshold = 5.0 bounce_info_stale_after = 7 bounce_you_are_disabled_warnings = 3 = 7
Am I right in thinking that if we make these values high enough, we'll see no accounts disabled, and the only side effects will be more bounces and yahoo mail won't get through? Would this be an acceptable solution for a list with only 1000 members and low traffic, assuming we warn the yahoo members to use a different address?
Just turn off bounce processing for the list. See the FAQ at <http://wiki.list.org/x/ggARAQ>.
Also consider what I speculate above in the paragraph starting with "Maybe."
Additional reading at <http://www.dmarc.org/faq.html#s_3>, <http://blog.threadable.com/how-threadable-solved-the-dmarc-problem> and <http://www.spamresource.com/2014/04/run-email-discussion-list-heres-how-to.h...> and other articles linked from those.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 12 Apr 2014, at 11:53 am, "Mark Sapiro" <mark@msapiro.net> wrote:
Maybe. Yahoo requests and receives reports of rejected mail. This is only speculation, but if Yahoo sees that your server is sending what it considers to be bogus mail purporting to be From: its domain, it could decide to reject all mail from your server.
That's a very disturbing thought, even more so it they're contributing to blocklists that are used elsewhere.
The list admin can see these values on the list's web admin Bounce processing page, but defaults are:
bounce_score_threshold = 5.0 bounce_info_stale_after = 7 bounce_you_are_disabled_warnings = 3 = 7
Thanks for those. Is the last one a typo? Otherwise, what does =3=7 mean?
With these settings, and address will have to bounce on 5 days, with no breaks of 7 days or more before being disabled?
After they're disabled, they get a warning email every bounce_you_are_disabled_warnings_interval days, correct? What's the default for that, please?
And then after bounce_you_are_disabled_warnings of these, they're actually unsubscribed?
Am I right in thinking that if we make these values high enough, we'll see no accounts disabled, and the only side effects will be more bounces and yahoo mail won't get through? Would this be an acceptable solution for a list with only 1000 members and low traffic, assuming we warn the yahoo members to use a different address?
Just turn off bounce processing for the list. See the FAQ at <http://wiki.list.org/x/ggARAQ>.
That should work for us for now, but won't we have a growing load of bounces as time goes by? I was thinking it might be better to get rid of those addresses that are permanently bouncing every message, even if we take longer to do it than before.
Additional reading at <http://www.dmarc.org/faq.html#s_3>, <http://blog.threadable.com/how-threadable-solved-the-dmarc-problem> and <http://www.spamresource.com/2014/04/run-email-discussion-list-heres-how-to.h...> and other articles linked from those.
From the threadable article: “He recommends that all list administrators immediately stop delivering to Yahoo addresses to limit damage, and encourage members to move to a more friendly provider."
How would not delivering to yahoo addresses help? I thought the problem was with delivering yahoo email to others.
And I'm wondering about asking people to move off yahoo. We might have to do that, but what happens if they go to the trouble of getting a gmail account, and then google starts doing the same thing? They're not going to be happy.
Peter Shute
On 04/12/2014 02:59 AM, Peter Shute wrote:
On 12 Apr 2014, at 11:53 am, "Mark Sapiro" <mark@msapiro.net> wrote:
bounce_you_are_disabled_warnings = 3 = 7
Thanks for those. Is the last one a typo? Otherwise, what does =3=7 mean?
It was supposed to say
bounce_you_are_disabled_warnings = 3 bounce_you_are_disabled_warnings_interval = 7
With these settings, and address will have to bounce on 5 days, with no breaks of 7 days or more before being disabled?
Correct.
After they're disabled, they get a warning email every bounce_you_are_disabled_warnings_interval days, correct? What's the default for that, please?
7 days, the bit that got dropped above.
And then after bounce_you_are_disabled_warnings of these, they're actually unsubscribed?
Yes.
Am I right in thinking that if we make these values high enough, we'll see no accounts disabled, and the only side effects will be more bounces and yahoo mail won't get through? Would this be an acceptable solution for a list with only 1000 members and low traffic, assuming we warn the yahoo members to use a different address?
Just turn off bounce processing for the list. See the FAQ at <http://wiki.list.org/x/ggARAQ>.
That should work for us for now, but won't we have a growing load of bounces as time goes by? I was thinking it might be better to get rid of those addresses that are permanently bouncing every message, even if we take longer to do it than before.
Yes, those are the tradeoffs you need to consider.
Additional reading at <http://www.dmarc.org/faq.html#s_3>, <http://blog.threadable.com/how-threadable-solved-the-dmarc-problem> and <http://www.spamresource.com/2014/04/run-email-discussion-list-heres-how-to.h...> and other articles linked from those.
From the threadable article: “He recommends that all list administrators immediately stop delivering to Yahoo addresses to limit damage, and encourage members to move to a more friendly provider."
How would not delivering to yahoo addresses help? I thought the problem was with delivering yahoo email to others.
See Jim P's post in this thread at <https://mail.python.org/pipermail/mailman-users/2014-April/076383.html> and the branch linked therefrom, although you probably don't have the access required to install it.
Jim P's approach is to reject any post From: a domain with a DMARC policy of reject.
To accept a post and then not deliver it to some because you know it won't be accepted is arguably wrong, and also, you can't know except by experience which recipient domains will reject the post for DMARC policy. I've seen the following:
aol.com att.net comcast.net compuserve.com hotmail.com msn.com netscape.net pacbell.net sbcglobal.net yahoo.com
and a buisiness domain hosted by Yahoo.
And I'm wondering about asking people to move off yahoo. We might have to do that, but what happens if they go to the trouble of getting a gmail account, and then google starts doing the same thing? They're not going to be happy.
It remains to be seen if DMARC becomes more widely adopted or dies, See Rich K's post in this thread at <https://mail.python.org/pipermail/mailman-users/2014-April/076392.html>. So far, Gmail/Googlemail is not honoring DMARC policy.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 12 Apr 2014, at 9:28 pm, "Mark Sapiro" <mark@msapiro.net> wrote:
Additional reading at <http://www.dmarc.org/faq.html#s_3>, <http://blog.threadable.com/how-threadable-solved-the-dmarc-problem> and <http://www.spamresource.com/2014/04/run-email-discussion-list-heres-how-to.h...> and other articles linked from those.
From the threadable article: “He recommends that all list administrators immediately stop delivering to Yahoo addresses to limit damage, and encourage members to move to a more friendly provider."
How would not delivering to yahoo addresses help? I thought the problem was with delivering yahoo email to others.
See Jim P's post in this thread at <https://mail.python.org/pipermail/mailman-users/2014-April/076383.html> and the branch linked therefrom, although you probably don't have the access required to install it.
Jim P's approach is to reject any post From: a domain with a DMARC policy of reject.
To accept a post and then not deliver it to some because you know it won't be accepted is arguably wrong, and also, you can't know except by experience which recipient domains will reject the post for DMARC policy. I've seen the following:
But note that the part of the threadable article I quoted talks about not *delivering* to yahoo addresses. I would thought that shouldn't be a problem.
Peter Shute
On Sat, Apr 12, 2014 at 6:15 PM, Peter Shute <pshute@nuw.org.au> wrote:
On 12 Apr 2014, at 9:28 pm, "Mark Sapiro" <mark@msapiro.net> wrote:
Additional reading at <http://www.dmarc.org/faq.html#s_3>, <http://blog.threadable.com/how-threadable-solved-the-dmarc-problem> and <http://www.spamresource.com/2014/04/run-email-discussion-list-heres-how-to.h...> and other articles linked from those.
From the threadable article: "He recommends that all list administrators immediately stop delivering to Yahoo addresses to limit damage, and encourage members to move to a more friendly provider."
How would not delivering to yahoo addresses help? I thought the problem was with delivering yahoo email to others.
See Jim P's post in this thread at <https://mail.python.org/pipermail/mailman-users/2014-April/076383.html> and the branch linked therefrom, although you probably don't have the access required to install it.
Jim P's approach is to reject any post From: a domain with a DMARC policy of reject.
To accept a post and then not deliver it to some because you know it won't be accepted is arguably wrong, and also, you can't know except by experience which recipient domains will reject the post for DMARC policy. I've seen the following:
But note that the part of the threadable article I quoted talks about not *delivering* to yahoo addresses. I would thought that shouldn't be a problem.
The recent yahoo change means that you (as a mailinglist operator) will have difficulties (and possibly eventually face blacklisting) by delivering, via your list, email that originates from a yahoo.com (and a few other) email address. Assuming you are doing SPF/DKIM/etc, you should have no problems delivering non-yahoo domain emails to yahoo.
-Jim P.
On 13 Apr 2014, at 8:20 am, "Jim Popovitch" <jimpop@gmail.com> wrote:
But note that the part of the threadable article I quoted talks about not *delivering* to yahoo addresses. I would thought that shouldn't be a problem.
The recent yahoo change means that you (as a mailinglist operator) will have difficulties (and possibly eventually face blacklisting) by delivering, via your list, email that originates from a yahoo.com (and a few other) email address. Assuming you are doing SPF/DKIM/etc, you should have no problems delivering non-yahoo domain emails to yahoo.
I don't know if we are doing SPF/DKIM ( or what they are). It's just a cpanel implementation of mailman. Are you suggesting we *could* have problems if we deliver non yahoo emails to our yahoo subscribers too?
Apologies for all these questions. I'm just trying to get my head around what this all means for our list.
Peter Shute
On Sat, Apr 12, 2014 at 8:17 PM, Peter Shute <pshute@nuw.org.au> wrote:
On 13 Apr 2014, at 8:20 am, "Jim Popovitch" <jimpop@gmail.com> wrote:
But note that the part of the threadable article I quoted talks about not *delivering* to yahoo addresses. I would thought that shouldn't be a problem.
The recent yahoo change means that you (as a mailinglist operator) will have difficulties (and possibly eventually face blacklisting) by delivering, via your list, email that originates from a yahoo.com (and a few other) email address. Assuming you are doing SPF/DKIM/etc, you should have no problems delivering non-yahoo domain emails to yahoo.
I don't know if we are doing SPF/DKIM ( or what they are). It's just a cpanel implementation of mailman. Are you suggesting we *could* have problems if we deliver non yahoo emails to our yahoo subscribers too?
No. You will be OK if you deliver non-yahoo email to yahoo (assuming that yahoo accepts email from you).
You *may* get into long term trouble (blacklisting) *IF* you deliver yahoo email to any other company/person that checks yahoo's DMARC record. Yahoo's recent change (DMARC p=reject) stipulates for everyone else to outright reject yahoo email from anyone other than yahoo. This has the potential for others to report your mailinglist as doing something malicious.
-Jim P.
Peter Shute writes:
I don't know if we are doing SPF/DKIM ( or what they are).
You should ask the people responsible for your mailserver. SPF and DKIM in themselves are good things because they prevent rejections of mail that you send directly to another domain that implements them, and because it's evidence to reasonable people that you follow best practice. If you/they are not doing it, you/they should.
How they work: SPF and DKIM are separate protocols that provide a certain degree of authentication for the *hosts* that transmit mail claiming to originate in a domain. The protocols work (more or less) by publishing a list of IP addresses that are allowed to send mail from the domain. Since it's information attached to a domain, you get that list from the domain's name server. There's a bit of crypto technology involved so that receivers can trust the information.
The problem is that the only IP address that you can trust at all is the direction connection from the host you receive the message from. In other words, although Internet mail is designed as a "store and forward" system where messages are passed from host to host until they reach their destination (where they user's mailbox is), effectively these protocols allow only one hop, or authentication fails.
In the case of DMARC (a "super" protocol that specifies how to use this information), a domain is allowed to *demand* that you reject the mail if authentication fails. That means that mailing lists (which necessarily involve at least two hops in most cases of interest) *always* fail authentication at *every* destination conforming to DMARC.
Yahoo! is lighting up a cigar in an elevator filled with pregnant women. :-( Fortunately, I suspect that they are about to bring down the wrath of Olympus upon themselves as their users start losing mail and being refused service on mailing lists, etc. This is a snafu on the order of Microsoft's backward compatibility break with Office '97 or so.
Regards, Steve
Our observation here has been that only Yahoo addresses, and those of other services which also uses the DMARC algorithm generate bounces. Because the From: address contains yahoo.com, and the IP address of the list server does not reverse resolve to a yahoo.com server, the list email is refused by Yahoo. The list of refusing servers includes Yahoo, Comcast, AT&T, Hotmail and a number of others.
Lindsay Haisley (512) 259-1190 (land line) (512) 496-7118 (mobile) Sent from my iPhone
On Apr 11, 2014, at 8:28 PM, Peter Shute <pshute@nuw.org.au> wrote:
I hadn't heard of this till now. Could somebody please confirm if my understanding of the issue is correct?
This is what I'm thinking will happen, please correct where I'm wrong:
- A list member sends an email to the list from a yahoo address
- The list sends that email out to all the list members
- The recipients' mail servers will (might?) check with yahoo what to do with the email, and will be advised to reject it
- The list will receive a bounce for every email address whose mail server follows that advice
- Those recipients whose mail server follows the advice will not receive the message
- The list will increment the bounce score for all those affected receipients, but only once per day
- The increment will be 1 because this is a hard bounce
- If the score reaches the bounce_score_threshold before the bounce_info_stale_after number of days has passed since the most recent bounce, then the member's subscription is disabled.
If that's correct then my understanding is that:
- If a list has at least one active yahoo member then pretty soon everyone's subscription will be disabled (not unsubscribed?).
- If a list receives vey few messages from yahoo addresses then the only effect will be that their messages don't get through, and that they might still get through to some people.
I'm a moderator for a cpanel list, but don't have access to any of the settings. Can someone tell me what the default settings are for bounce_score_threshold and bounce_info_stale_after? I'm assuming ours might still be whatever the defaults are.
Am I right in thinking that if we make these values high enough, we'll see no accounts disabled, and the only side effects will be more bounces and yahoo mail won't get through? Would this be an acceptable solution for a list with only 1000 members and low traffic, assuming we warn the yahoo members to use a different address?
Peter Shute
Siniša Burina wrote: I believe there's no need to elaborate on the problems recently introduced by Yahoo, changing their DMARC DNS record and rendering many mailman lists unusable for Yahoo mail users.
Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/fmouse%40fmp.com
participants (12)
-
Adam McGreggor -
Barry Warsaw -
Jim Popovitch -
Joseph Brennan -
Lindsay Haisley -
mail.ulticom.com -
Mark Sapiro -
Mitra IMAP
-
Peter Shute -
Rich Kulawiec -
Siniša Burina -
Stephen J. Turnbull