Dealing with rate limiting from Roadrunner/Time Warner
Has anyone had to deal with bounces due to rate limiting from Roadrunner/Time Warner?
http://postmaster.rr.com/spam#ratelimit
If so, what did you do to resolve the problem? (That is, if things “got resolved.”)
My hosting provider, DreamHost, continues to assert this is my problem and not theirs. I of course do not have control of the Mailman installation and/or the server.
-Conrad
-- Is there a suspect in your family? Contact the Ministry of Information. Ring 100 00 00.
Conrad G T Yoder writes:
Has anyone had to deal with bounces due to rate limiting from Roadrunner/Time Warner?
Are these true bounces (ie, permanent delivery failures) or just the temporary failures due to rate limiting, causing delays of many hours or days in delivery?
Haven't had a problem (RR is on the list of "if you don't like losing mail, don't do that" mailboxes for my lists).
My hosting provider, DreamHost, continues to assert this is my problem and not theirs.
This may be partly true, if you are falling afoul of the "too many recipients per message" limit. In that case, if the option is available, then you should set personalization on for lists will more than a very few RR subscribers. If not, see "full of" below.
I of course do not have control of the Mailman installation and/or the server.
NightmareHost is full of s--t. The fundamental problem is RR's policy. They implicitly say "we don't care if our users get their mail,"[1] and they reserve the right to arbitrarily refuse delivery no matter what you do. That may be your problem, but there's nothing you can do about it.
It's true that you *may* have a mitigation option in personalization, but that is under control of NightmareHost. If they do not permit you to personalize, there's nothing under your control that you can do. All the other mitigation strategies (whitelisting, feedback loop) involve the participation of NightmareHost AFAICS, since RR rate limits on the basis of IP, which is owned by NightmareHost, I assume, and the limit on recipients per SMTP connection is in mm_cfg.py, which is not under your control at all.
Does NightmareHost offer you any advice? Or are they just saying "if you don't like our service, find another one?" I think "find another one" is a great idea, personally.
Sorry I can't be more optimistic.
Footnotes: [1] This is generally true of the big services. They deliberately accept the tradeoff of unreliable delivery of desired mail in return for less spam. They prefer not to spin it that way, but that's what they do.
On May 15, 2014, at 12:21 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
It is a true bounce - mail is being rejected. The error message is phrased as a "temporary failure," but the message is bounced at the SMTP transaction. “At some point,” they stop bouncing. RR won’t say when that happens.
<user@roadrunner.com>: delivery temporarily suspended: host cdptpa-pub-iedge-vip.email.rr.com[107.14.166.70] refused to talk to me: 421 4.7.1 - Connection refused - <66.33.216.56> - Too many concurrent connections ( <2> ) from source IP
I will probably turn that on soon - it is available to me.
On this instance, they haven’t been very helpful. For the most part, though, they have been decent. The largest of my lists that I admin has over 7600 subscribers, with about 15 emails going out daily - that’s about 115K total emails a day, and for the price I am paying DH, I haven’t found anywhere near a better deal. So I accept these (occasional) issues along with the deal I’m getting.
-Conrad
-- See something, Say something.
On 05/15/2014 06:46 AM, Conrad G T Yoder wrote:
This is a 4xx status, so it is a temp failure, and Mailman should be retrying these.
It looks like what is going on is the outgoing MTA has received multiple messages from Mailman for RR recipients, and is trying to send them in parallel.
The bottom line is that these messages should be ultimately delivered.
Note that if my analysis is correct, it would appear that Mailman is already sending 1 message per recipient to the outgoing MTA because of personalization or VERP enabled or SMTP_MAX_RCPTS = 1, and this is a case where a larger SMTP_MAX_RCPTS and no VERP or personalization might help.
Unfortunately, the only one of these things that *might* be available to a list owner is personalization.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Conrad G T Yoder writes:
The log you display is not a true bounce. If your host's MTA is configured properly, you may get DSNs saying that you don't need to do anything but wait, but your users' bounce counts are not increasing due to messages like the one below, and they probably are eventually being delivered (although that can't be determined from the log below).
This is absolutely clearly with no doubt whatsoever a problem that only can be resolved by reconfiguring the MTA at one end or the other. I gather that your MTA is a DreamHost responsibility, so they are either amazingly incompetent or lying to you. I don't blame them for not wanting to deal with it, by the way, only for trying to blame the victims (you and your Roadrunner subscribers). Roadrunner's policy is brain-damaged, DreamHost's is merely pragmatic (it's very costly to try to work around a peer site's brain damage).
The only effective measures you can take that I can see based on what you have written so far are (1) find a competent and responsive host for your list (there may not be any, though; I'm not sure what to do to get reliable timely delivery to a system like Roadrunner), or (2) tell your Roadrunner subscribers that their host is incapable of supporting reliable mail delivery and that you can't do anything; if they care about reliable delivery of mailing lists including yours, they should use an address at a competent provider (GMail is the only competent and approximately socially responsible freemail provider I know of).
This probably won't help, because based on the message above your MTA will helpfully just execute the deliveries in parallel, and you'll get *more* delivery failures, not less.
This issue doesn't look "occasional" to me. However, if what is most likely a permanent inability to achieve timely delivery to subscribers at Roadrunner is acceptable to you, I don't have any complaints. :-/
Steve
On May 15, 2014, at 8:48 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
Gotcha. I guess someone thought it was a true bounce and configured their servers appropriately. :^)
I will certainly be encouraging people with RR addresses to switch.
I have not yet tried the nuclear option - the infamous EECB (Executive Email Carpet Bomb). This of course should only be used when all other options have been exhausted. I’ll probably end up doing it with the Time Warner people. I did it a few years ago when I was (having no success) going through the usual channels of AOL marking mailing list email as spam. Within 2 hours of the EECB I got a call from one of their Anti-Spam Operations team members, and she gave me her direct phone number and email address at that point for any future needed correspondence. Things started happening then.
-Conrad
-- Truth is information.
Conrad G T Yoder writes:
On May 15, 2014, at 8:48 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
Could be.
The other possibility is that you have enough Roadrunner subscribers that when a post goes out, one connection gets through, a bunch get a temporary fail and requeued. 4 hours later, your MTA tries a bunch and one gets through. A bunch minus 1 get a temporary fail, and requeued. 8 hours later, your MTA tries a bunch minus 1 and one gets through. A bunch minus 2 get a temporary fail, and requeued. This happens up to 5 times with approximately exponential backoff. If the 5th retry fails, it will be considered a permanent failure (by your MTA). So if "a bunch" is bigger than 5, you'll get "true bounces". If any of these retries overlaps with a new post, the problem gets compounded. (All the concrete numbers depend on your MTA's configuration, but they're pretty typical, I believe).
At this point, it should be clear why I consider Roadrunner's policy to be braindamaged. They're wasting your resources and DoS'ing their own users.
BTW, it should be possible to reconfigure your Mailman and MTA so that Mailman sends the maximum number of addresses per SMTP connection (for this you need to turn personalization OFF) permitted by Roadrunner in one shot, and the MTA does as well. It may be possible to get your MTA or Mailman to sequence contacts to Roadrunner so that only one SMTP connection is open at a time. Maybe Roadrunner will tell you what the relevant numbers are, or even give you a higher limit.
Nevertheless, as long as they have this policy, the possibility of a retry cascade exists. And the numbers that Roadrunner offers may be much smaller than your MTA's optimal configuration from your point of view.
I have not yet tried the nuclear option - the infamous EECB (Executive Email Carpet Bomb).
Bu-wha-ha-ha-ha! But I hope things don't go that far.
Steve
Conrad G T Yoder writes:
Has anyone had to deal with bounces due to rate limiting from Roadrunner/Time Warner?
Are these true bounces (ie, permanent delivery failures) or just the temporary failures due to rate limiting, causing delays of many hours or days in delivery?
Haven't had a problem (RR is on the list of "if you don't like losing mail, don't do that" mailboxes for my lists).
My hosting provider, DreamHost, continues to assert this is my problem and not theirs.
This may be partly true, if you are falling afoul of the "too many recipients per message" limit. In that case, if the option is available, then you should set personalization on for lists will more than a very few RR subscribers. If not, see "full of" below.
I of course do not have control of the Mailman installation and/or the server.
NightmareHost is full of s--t. The fundamental problem is RR's policy. They implicitly say "we don't care if our users get their mail,"[1] and they reserve the right to arbitrarily refuse delivery no matter what you do. That may be your problem, but there's nothing you can do about it.
It's true that you *may* have a mitigation option in personalization, but that is under control of NightmareHost. If they do not permit you to personalize, there's nothing under your control that you can do. All the other mitigation strategies (whitelisting, feedback loop) involve the participation of NightmareHost AFAICS, since RR rate limits on the basis of IP, which is owned by NightmareHost, I assume, and the limit on recipients per SMTP connection is in mm_cfg.py, which is not under your control at all.
Does NightmareHost offer you any advice? Or are they just saying "if you don't like our service, find another one?" I think "find another one" is a great idea, personally.
Sorry I can't be more optimistic.
Footnotes: [1] This is generally true of the big services. They deliberately accept the tradeoff of unreliable delivery of desired mail in return for less spam. They prefer not to spin it that way, but that's what they do.
On May 15, 2014, at 12:21 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
It is a true bounce - mail is being rejected. The error message is phrased as a "temporary failure," but the message is bounced at the SMTP transaction. “At some point,” they stop bouncing. RR won’t say when that happens.
<user@roadrunner.com>: delivery temporarily suspended: host cdptpa-pub-iedge-vip.email.rr.com[107.14.166.70] refused to talk to me: 421 4.7.1 - Connection refused - <66.33.216.56> - Too many concurrent connections ( <2> ) from source IP
I will probably turn that on soon - it is available to me.
On this instance, they haven’t been very helpful. For the most part, though, they have been decent. The largest of my lists that I admin has over 7600 subscribers, with about 15 emails going out daily - that’s about 115K total emails a day, and for the price I am paying DH, I haven’t found anywhere near a better deal. So I accept these (occasional) issues along with the deal I’m getting.
-Conrad
-- See something, Say something.
On 05/15/2014 06:46 AM, Conrad G T Yoder wrote:
This is a 4xx status, so it is a temp failure, and Mailman should be retrying these.
It looks like what is going on is the outgoing MTA has received multiple messages from Mailman for RR recipients, and is trying to send them in parallel.
The bottom line is that these messages should be ultimately delivered.
Note that if my analysis is correct, it would appear that Mailman is already sending 1 message per recipient to the outgoing MTA because of personalization or VERP enabled or SMTP_MAX_RCPTS = 1, and this is a case where a larger SMTP_MAX_RCPTS and no VERP or personalization might help.
Unfortunately, the only one of these things that *might* be available to a list owner is personalization.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Conrad G T Yoder writes:
The log you display is not a true bounce. If your host's MTA is configured properly, you may get DSNs saying that you don't need to do anything but wait, but your users' bounce counts are not increasing due to messages like the one below, and they probably are eventually being delivered (although that can't be determined from the log below).
This is absolutely clearly with no doubt whatsoever a problem that only can be resolved by reconfiguring the MTA at one end or the other. I gather that your MTA is a DreamHost responsibility, so they are either amazingly incompetent or lying to you. I don't blame them for not wanting to deal with it, by the way, only for trying to blame the victims (you and your Roadrunner subscribers). Roadrunner's policy is brain-damaged, DreamHost's is merely pragmatic (it's very costly to try to work around a peer site's brain damage).
The only effective measures you can take that I can see based on what you have written so far are (1) find a competent and responsive host for your list (there may not be any, though; I'm not sure what to do to get reliable timely delivery to a system like Roadrunner), or (2) tell your Roadrunner subscribers that their host is incapable of supporting reliable mail delivery and that you can't do anything; if they care about reliable delivery of mailing lists including yours, they should use an address at a competent provider (GMail is the only competent and approximately socially responsible freemail provider I know of).
This probably won't help, because based on the message above your MTA will helpfully just execute the deliveries in parallel, and you'll get *more* delivery failures, not less.
This issue doesn't look "occasional" to me. However, if what is most likely a permanent inability to achieve timely delivery to subscribers at Roadrunner is acceptable to you, I don't have any complaints. :-/
Steve
On May 15, 2014, at 8:48 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
Gotcha. I guess someone thought it was a true bounce and configured their servers appropriately. :^)
I will certainly be encouraging people with RR addresses to switch.
I have not yet tried the nuclear option - the infamous EECB (Executive Email Carpet Bomb). This of course should only be used when all other options have been exhausted. I’ll probably end up doing it with the Time Warner people. I did it a few years ago when I was (having no success) going through the usual channels of AOL marking mailing list email as spam. Within 2 hours of the EECB I got a call from one of their Anti-Spam Operations team members, and she gave me her direct phone number and email address at that point for any future needed correspondence. Things started happening then.
-Conrad
-- Truth is information.
Conrad G T Yoder writes:
On May 15, 2014, at 8:48 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
Could be.
The other possibility is that you have enough Roadrunner subscribers that when a post goes out, one connection gets through, a bunch get a temporary fail and requeued. 4 hours later, your MTA tries a bunch and one gets through. A bunch minus 1 get a temporary fail, and requeued. 8 hours later, your MTA tries a bunch minus 1 and one gets through. A bunch minus 2 get a temporary fail, and requeued. This happens up to 5 times with approximately exponential backoff. If the 5th retry fails, it will be considered a permanent failure (by your MTA). So if "a bunch" is bigger than 5, you'll get "true bounces". If any of these retries overlaps with a new post, the problem gets compounded. (All the concrete numbers depend on your MTA's configuration, but they're pretty typical, I believe).
At this point, it should be clear why I consider Roadrunner's policy to be braindamaged. They're wasting your resources and DoS'ing their own users.
BTW, it should be possible to reconfigure your Mailman and MTA so that Mailman sends the maximum number of addresses per SMTP connection (for this you need to turn personalization OFF) permitted by Roadrunner in one shot, and the MTA does as well. It may be possible to get your MTA or Mailman to sequence contacts to Roadrunner so that only one SMTP connection is open at a time. Maybe Roadrunner will tell you what the relevant numbers are, or even give you a higher limit.
Nevertheless, as long as they have this policy, the possibility of a retry cascade exists. And the numbers that Roadrunner offers may be much smaller than your MTA's optimal configuration from your point of view.
I have not yet tried the nuclear option - the infamous EECB (Executive Email Carpet Bomb).
Bu-wha-ha-ha-ha! But I hope things don't go that far.
Steve
participants (3)
-
Conrad G T Yoder -
Mark Sapiro -
Stephen J. Turnbull