Warming IP Addresses -- Gradually Increasing Email Volume

We have three mailman lists with over 20K members. The lists are subscription based which distribute content to members by email each morning.
We need to migrate from our current server to a new one. We will need to change IP addresses as part of the migration. This video suggests sending small volumes of email initially and building the volume over time:
https://sendgrid.com/docs/Classroom/Deliver/Delivery_Introduction/warming_up...
I'm sure sending 20K emails on day one would not make the ISPs very happy.
Does anyone know a way to warm up an IP Address while using mailman? We need to have a plan for this to complete our server migration plan.
Thanks, Greg

On 2/20/2017 10:01 PM, Greg Sims wrote:
Have you talked to the ISP? The environment and connections also matter. One server on the end of a residential DSL is a bit different from a hosted instance from a large corporate system.
If nothing else and assuming a moderate business or academic environment, the mailman server would be sending everything to a local relay server (postfix/sendmail/exim/etc) and that would be sending the mail to it's destinations. If the relay server is at the ISP, definitely talk to them first; I don't think "warming" a connection will help. (And if you have your own relay and send to all destinations from there, the ISP's server isn't in the picture and they shouldn't care about the traffic.)
YMMV, of course.
Later,
z!

Thanks for your feedback Carl. Here is some additional information about our environment. We have a single server in a commercial data center (Softlayer). "Communicating" to the ISPs is a challenge. No reply to email and no way to discuss something like this on the telephone. The only answer is to construct a strategy that does not raise a "red flag".
---------- Forwarded message ---------- From: Carl Zwanzig <cpz@tuunq.com> Date: Tue, Feb 21, 2017 at 10:07 PM Subject: Re: [Mailman-Users] Warming IP Addresses -- Gradually Increasing Email Volume To: mailman-users@python.org
On 2/20/2017 10:01 PM, Greg Sims wrote:
[...]
I'm sure sending 20K emails on day one would not make the ISPs very happy.
Have you talked to the ISP? The environment and connections also matter. One server on the end of a residential DSL is a bit different from a hosted instance from a large corporate system.
If nothing else and assuming a moderate business or academic environment, the mailman server would be sending everything to a local relay server (postfix/sendmail/exim/etc) and that would be sending the mail to it's destinations. If the relay server is at the ISP, definitely talk to them first; I don't think "warming" a connection will help. (And if you have your own relay and send to all destinations from there, the ISP's server isn't in the picture and they shouldn't care about the traffic.)
YMMV, of course.
Later,
z!
Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/greg% 40headingup.net

Greg Sims <greg@headingup.net> writes:
Greg,
I would think that a service provider that does not reply to emails in a timely manner and does not provide telephone support is, in itself, a "red flag" that this ISP does not want your business and does not care about your reputation or theirs. I would start looking for a provider that provides support and is willing to communicate.
Keith
--
from my mac to yours...
Keith Seyffarth mailto:weif@weif.net http://www.weif.net/ - Home of the First Tank Guide! http://www.rpgcalendar.net/ - the Montana Role-Playing Calendar
http://www.miscon.org/ - Montana's Longest Running Science Fiction Convention

Hey Keith,
The communications I have with my ISP (Softlayer) is fine. The issue is with AOL, Yahoo and others. These folks are impossible to communication with.
Greg
On Wed, Feb 22, 2017 at 7:23 AM, Keith Seyffarth <weif@weif.net> wrote:

On 2/22/2017 7:33 AM, Greg Sims wrote:
Yes they are and there's not much to be done about it (issues with aol & yahoo have been covered extensively on this list, please check the archives). It was not clear which ISP you were referring to so everyone assumes the closest one.
Assuming that mailman delivers to a real MTA (postfix/sendmail/etc) on your server and that sends the mail out, you need to make sure the new server/IP has proper a proper DKIM/SPF config and then look at the whitelist procedures at aol/yahoo/etc. You also need the latest mailman (2.1.23) with DMARC mitigation configs (also discussed on this list).
Later,
z!

Hi Craig
Are you already using DMARC / DKIM to sign all messages? One of the mailop people from a big mail provider mentioned that if you have this configured and it has a reputation then provided you retain the same key and the other details the same the reputation you have will be tracked and follow that along.
This obviously requires this configuration to be setup however if you have some time left before you decommission then you could get this in place to attract your existing reputation to the key and help (nothing is perfect but all of this is a numbers game)
While I haven't seen this talked about specifically on top of the above (in a belt and suspenders kind of way) keeping the same forward / reverse / EHLO name for your new server as the old (provided the old has these configured to your domain) this could provide another signal that this is the same system doing the same job just moved to a new IP Address.
Nothing here will be perfect and if you have the time warming up the new IP (perhaps setting up the new server's MTA possibly needing different name in this scenario) and asking your existing server to pass off a small percentage of your email to that machine as an outbound relay and increase the number until close to all via the new server then complete the migration.
I don't have a guide on the above myself as I haven't had to do this. But from all the advice I have seen on lists about this topic they seem to be ideas that will aid deliver ability.

The concerns Greg has are real. For example, a little over a year ago, the server mail.python.org that hosts over 300 @python.org lists (including this one) suffered a fatal hardware error on a Friday. With much scrambling, we were able to get everything up by the next Monday on a new server in the cloud, but with a different IP.
It was several weeks before we were able to set up feedback loops and such with the various services such as AOL, Gmail, etc., which knew our prior IP and get them to accept mail from the new IP without bounces, greylisting, etc. One thing we did observe is if possible, when the ISP has an IPv6 MX, make sure your outging MTA can use it as at least some of the ISPs are more lenient about what they accept via IPv6.
Also, see <https://wiki.list.org/x/4030690> about signing up for feedback loop reporting with the various ISPs. If you can do this in advance for the new IP, this will help.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

The communications I have with my ISP (Softlayer) is fine. The issue is with AOL, Yahoo and others. These folks are impossible to communication with.
That is much more understandable.
If you want to slow the rate of outgoing messages, talk to Softlayer about possibly throttling your send rate for a while. How long will depend on how often mailings are sent and how many of your subscribers are on any given mailhost. If you really don't have a lot of subscribers all on hotmail or all on Yahoo! or all on Gmail, then the throttling is probably less of an issue.
Also check the current reputation of your sending IP address. If it already has a developed good rating, and your messages are unlikely to be flagged with a little more traffic from your mailing list.
Keith
--
from my mac to yours...
Keith Seyffarth mailto:weif@weif.net http://www.weif.net/ - Home of the First Tank Guide! http://www.rpgcalendar.net/ - the Montana Role-Playing Calendar
http://www.miscon.org/ - Montana's Longest Running Science Fiction Convention

On 2/20/2017 10:01 PM, Greg Sims wrote:
Have you talked to the ISP? The environment and connections also matter. One server on the end of a residential DSL is a bit different from a hosted instance from a large corporate system.
If nothing else and assuming a moderate business or academic environment, the mailman server would be sending everything to a local relay server (postfix/sendmail/exim/etc) and that would be sending the mail to it's destinations. If the relay server is at the ISP, definitely talk to them first; I don't think "warming" a connection will help. (And if you have your own relay and send to all destinations from there, the ISP's server isn't in the picture and they shouldn't care about the traffic.)
YMMV, of course.
Later,
z!

Thanks for your feedback Carl. Here is some additional information about our environment. We have a single server in a commercial data center (Softlayer). "Communicating" to the ISPs is a challenge. No reply to email and no way to discuss something like this on the telephone. The only answer is to construct a strategy that does not raise a "red flag".
---------- Forwarded message ---------- From: Carl Zwanzig <cpz@tuunq.com> Date: Tue, Feb 21, 2017 at 10:07 PM Subject: Re: [Mailman-Users] Warming IP Addresses -- Gradually Increasing Email Volume To: mailman-users@python.org
On 2/20/2017 10:01 PM, Greg Sims wrote:
[...]
I'm sure sending 20K emails on day one would not make the ISPs very happy.
Have you talked to the ISP? The environment and connections also matter. One server on the end of a residential DSL is a bit different from a hosted instance from a large corporate system.
If nothing else and assuming a moderate business or academic environment, the mailman server would be sending everything to a local relay server (postfix/sendmail/exim/etc) and that would be sending the mail to it's destinations. If the relay server is at the ISP, definitely talk to them first; I don't think "warming" a connection will help. (And if you have your own relay and send to all destinations from there, the ISP's server isn't in the picture and they shouldn't care about the traffic.)
YMMV, of course.
Later,
z!
Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/greg% 40headingup.net

Greg Sims <greg@headingup.net> writes:
Greg,
I would think that a service provider that does not reply to emails in a timely manner and does not provide telephone support is, in itself, a "red flag" that this ISP does not want your business and does not care about your reputation or theirs. I would start looking for a provider that provides support and is willing to communicate.
Keith
--
from my mac to yours...
Keith Seyffarth mailto:weif@weif.net http://www.weif.net/ - Home of the First Tank Guide! http://www.rpgcalendar.net/ - the Montana Role-Playing Calendar
http://www.miscon.org/ - Montana's Longest Running Science Fiction Convention

Hey Keith,
The communications I have with my ISP (Softlayer) is fine. The issue is with AOL, Yahoo and others. These folks are impossible to communication with.
Greg
On Wed, Feb 22, 2017 at 7:23 AM, Keith Seyffarth <weif@weif.net> wrote:

On 2/22/2017 7:33 AM, Greg Sims wrote:
Yes they are and there's not much to be done about it (issues with aol & yahoo have been covered extensively on this list, please check the archives). It was not clear which ISP you were referring to so everyone assumes the closest one.
Assuming that mailman delivers to a real MTA (postfix/sendmail/etc) on your server and that sends the mail out, you need to make sure the new server/IP has proper a proper DKIM/SPF config and then look at the whitelist procedures at aol/yahoo/etc. You also need the latest mailman (2.1.23) with DMARC mitigation configs (also discussed on this list).
Later,
z!

Hi Craig
Are you already using DMARC / DKIM to sign all messages? One of the mailop people from a big mail provider mentioned that if you have this configured and it has a reputation then provided you retain the same key and the other details the same the reputation you have will be tracked and follow that along.
This obviously requires this configuration to be setup however if you have some time left before you decommission then you could get this in place to attract your existing reputation to the key and help (nothing is perfect but all of this is a numbers game)
While I haven't seen this talked about specifically on top of the above (in a belt and suspenders kind of way) keeping the same forward / reverse / EHLO name for your new server as the old (provided the old has these configured to your domain) this could provide another signal that this is the same system doing the same job just moved to a new IP Address.
Nothing here will be perfect and if you have the time warming up the new IP (perhaps setting up the new server's MTA possibly needing different name in this scenario) and asking your existing server to pass off a small percentage of your email to that machine as an outbound relay and increase the number until close to all via the new server then complete the migration.
I don't have a guide on the above myself as I haven't had to do this. But from all the advice I have seen on lists about this topic they seem to be ideas that will aid deliver ability.

Thank you Alexander! Yes, DMARC/DKIM has been up and running for years now. You have some good ideas here.
On Wed, Feb 22, 2017 at 8:28 AM, Alexander Neilson <alexander@neilson.net.nz
wrote:

The concerns Greg has are real. For example, a little over a year ago, the server mail.python.org that hosts over 300 @python.org lists (including this one) suffered a fatal hardware error on a Friday. With much scrambling, we were able to get everything up by the next Monday on a new server in the cloud, but with a different IP.
It was several weeks before we were able to set up feedback loops and such with the various services such as AOL, Gmail, etc., which knew our prior IP and get them to accept mail from the new IP without bounces, greylisting, etc. One thing we did observe is if possible, when the ISP has an IPv6 MX, make sure your outging MTA can use it as at least some of the ISPs are more lenient about what they accept via IPv6.
Also, see <https://wiki.list.org/x/4030690> about signing up for feedback loop reporting with the various ISPs. If you can do this in advance for the new IP, this will help.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

The communications I have with my ISP (Softlayer) is fine. The issue is with AOL, Yahoo and others. These folks are impossible to communication with.
That is much more understandable.
If you want to slow the rate of outgoing messages, talk to Softlayer about possibly throttling your send rate for a while. How long will depend on how often mailings are sent and how many of your subscribers are on any given mailhost. If you really don't have a lot of subscribers all on hotmail or all on Yahoo! or all on Gmail, then the throttling is probably less of an issue.
Also check the current reputation of your sending IP address. If it already has a developed good rating, and your messages are unlikely to be flagged with a little more traffic from your mailing list.
Keith
--
from my mac to yours...
Keith Seyffarth mailto:weif@weif.net http://www.weif.net/ - Home of the First Tank Guide! http://www.rpgcalendar.net/ - the Montana Role-Playing Calendar
http://www.miscon.org/ - Montana's Longest Running Science Fiction Convention
participants (5)
-
Alexander Neilson
-
Carl Zwanzig
-
Greg Sims
-
Keith Seyffarth
-
Mark Sapiro