At Tue, 23 Dec 2014 09:53:13 +1100 "Steven D'Aprano" <steve@pearwood.info> wrote:

Hi all,

Back in September, I migrated an announcement-only mailing list from Yahoo to mailman, with approximately 1800 email addresses. An unknown number of these were dead.

The mailing list I am running sends out one, maybe two, emails a month, at around the same time of the month. It's not a discussion list, so there are no replies from subscribers.

I expected that the first month, there would be a lot of automatic unsubscriptions, as the dead addresses were noted and removed, and then in subsequent months there might only be a trickle of automatic removals. To my surprise though, there has been a steady pattern of mass unsubscriptions each month, around 100-200 each month following the posted announcement.

I'm not sure whether this is normal (if it is, unless the number of unsubscriptions begins to fall soon, I'll soon be left with no subscribers) or whether I'm doing something wrong.

Are these mass removals automatic or manual? That is, is the list doing this or are your subscribers doing it?

Some notes:

1. What are you doing about DMARC, DKIM, and ADSP? Remember, *Yahoo* has a strict reject DMARC policy. Of course this really kicks in if a Yahoo person posts. It *you* are the only poster and are posting from a non-Yahoo address (that is from a domain that does not have a strict reject DMARC policy), then this does not apply.

2. YahooGroups *does not* (AFAIK) send out monthly reminders. Mailman does.
   Your Yahoo users might be thinking the Mailman monthly reminder is some sort of robotic spam message and rejecting it.

3. It is possible that many of your Yahoo users are actually alive and well, but that YahooGroups suspended mail delivery of the YahooGroups mailings. Now that you switched to Mailman, these users are 'suddenly' getting these messages from a mailing list they have completely forgotten about.

Here are my bounce processing settings:

bounce_processing = Yes bounce_score_threshold = 4.0 bounce_info_stale_after = 65 bounce_you_are_disabled_warnings = 3 bounce_you_are_disabled_warnings_interval = 10

Remembering that I only send out 1 or 2 emails every 30 days (give or take a couple of days in either direction), does this seem reasonable? Or is my bounce processing too strict?

Thanks in advance,

-- Robert Heller -- 978-544-6933 Deepwoods Software -- Custom Software Services http://www.deepsoft.com/ -- Linux Administration Services heller@deepsoft.com -- Webhosting Services

On 12/22/2014 02:53 PM, Steven D'Aprano wrote:

Back in September, I migrated an announcement-only mailing list from Yahoo to mailman, with approximately 1800 email addresses. An unknown number of these were dead.

Doesn't Yahoo process bounces from dead addresses :(

Here are my bounce processing settings:

bounce_processing = Yes bounce_score_threshold = 4.0 bounce_info_stale_after = 65 bounce_you_are_disabled_warnings = 3 bounce_you_are_disabled_warnings_interval = 10

Remembering that I only send out 1 or 2 emails every 30 days (give or take a couple of days in either direction), does this seem reasonable? Or is my bounce processing too strict?

I don't think it's too strict. I might actually lower bounce_score_threshold from 4.0 to 3.0, but here's what I would expect with your settings:

A user has to bounce on 4 separate days, i.e. 4 messages in your case, with less than 65 days between bounces, at which point the user's bounce score will reach 4 and the user's delivery will be disabled by bounce and if bounce_notify_owner_on_disable is Yes, you will get a notice of the disable with the triggering bounce attached.

This won't happen until you've sent at least 4 messages to the list, i.e. you won't see anything for 2 to 4 months.

When the user's delivery is disabled, she is sent a warning, and then two more at 10 day intervals and then is removed after 10 more days, i.e. 30 days after delivery was disabled. If bounce_notify_owner_on_removal is Yes, you will be notified of this.

Note that the unsubscribes are not directly tied to a recent post. Unsubscribes occur because the user's delivery was disabled 30 days prior.

Bounce scoring and disabling does not necessarily occur immediately after a post. In many cases it will because the recipients MX will not accept the message from the Mailman server, but in other cases, errors may cause delayed delivery which doesn't become fatal the a few days. In other cases, there are several hops to the ultimate delivery, and the bounce DSN doesn't make it back to the Mailman server for up to a few days.

Still, I wouldn't expect a steady incremental removal such as you describe.

Note that things may not be as I describe. The behavior I describe is correct for all Mailman 2.1.x versions except 2.1.5 and 2.1.6+ for sites which have set VERP_PROBES = Yes in mm_cfg.py (Pre 2.1.5 there were no VERP probes, in 2.1.5 VERP_PROBES = Yes was the default, since that release the default is VERP_PROBES = No).

If VERP_PROBES is Yes, when a users bounce score reaches the threshold, the user's score is reset and the user is sent a probe message with a VERPed envelope sender, and only if and when the probe bounces is the user's delivery disabled.

Note also that the sending of all but the first warning to the user and the ultimate unsubscription is under control of Mailman's cron/disabled and won't occur if that cron is not being run.

If you still have questions about this, tell us what Mailman version this is, whether or not you are receiving bounce_notify_owner_on_disable notices (if it's Off, turn it On), how they correlate with the unsubscribes and whether the attached DSNs look like real bounces, and if you have access to Mailman's logs, what's in the 'bounce' log going back as far as you can.

Note that it is possible, though I would think unlikely in the volume you notice, the users don't bounce every message, but bounce often enough to have delivery ultimately disabled.

It occurs to me that a much more likely scenario is you are on a hosted Mailman instance or using a 'smarthost' outgoing MTA, and some fraction of every list post is being bounced because you are running afoul of some recipient limit/rate limit imposed by the host.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

At Tue, 23 Dec 2014 09:53:13 +1100 "Steven D'Aprano" <steve@pearwood.info> wrote:

Hi all,

Back in September, I migrated an announcement-only mailing list from Yahoo to mailman, with approximately 1800 email addresses. An unknown number of these were dead.

The mailing list I am running sends out one, maybe two, emails a month, at around the same time of the month. It's not a discussion list, so there are no replies from subscribers.

I expected that the first month, there would be a lot of automatic unsubscriptions, as the dead addresses were noted and removed, and then in subsequent months there might only be a trickle of automatic removals. To my surprise though, there has been a steady pattern of mass unsubscriptions each month, around 100-200 each month following the posted announcement.

I'm not sure whether this is normal (if it is, unless the number of unsubscriptions begins to fall soon, I'll soon be left with no subscribers) or whether I'm doing something wrong.

Are these mass removals automatic or manual? That is, is the list doing this or are your subscribers doing it?

Some notes:

1. What are you doing about DMARC, DKIM, and ADSP? Remember, *Yahoo* has a strict reject DMARC policy. Of course this really kicks in if a Yahoo person posts. It *you* are the only poster and are posting from a non-Yahoo address (that is from a domain that does not have a strict reject DMARC policy), then this does not apply.

2. YahooGroups *does not* (AFAIK) send out monthly reminders. Mailman does.
   Your Yahoo users might be thinking the Mailman monthly reminder is some sort of robotic spam message and rejecting it.

3. It is possible that many of your Yahoo users are actually alive and well, but that YahooGroups suspended mail delivery of the YahooGroups mailings. Now that you switched to Mailman, these users are 'suddenly' getting these messages from a mailing list they have completely forgotten about.

Here are my bounce processing settings:

bounce_processing = Yes bounce_score_threshold = 4.0 bounce_info_stale_after = 65 bounce_you_are_disabled_warnings = 3 bounce_you_are_disabled_warnings_interval = 10

Remembering that I only send out 1 or 2 emails every 30 days (give or take a couple of days in either direction), does this seem reasonable? Or is my bounce processing too strict?

Thanks in advance,

-- Robert Heller -- 978-544-6933 Deepwoods Software -- Custom Software Services http://www.deepsoft.com/ -- Linux Administration Services heller@deepsoft.com -- Webhosting Services

On 12/22/2014 02:53 PM, Steven D'Aprano wrote:

Back in September, I migrated an announcement-only mailing list from Yahoo to mailman, with approximately 1800 email addresses. An unknown number of these were dead.

Doesn't Yahoo process bounces from dead addresses :(

Here are my bounce processing settings:

bounce_processing = Yes bounce_score_threshold = 4.0 bounce_info_stale_after = 65 bounce_you_are_disabled_warnings = 3 bounce_you_are_disabled_warnings_interval = 10

Remembering that I only send out 1 or 2 emails every 30 days (give or take a couple of days in either direction), does this seem reasonable? Or is my bounce processing too strict?

I don't think it's too strict. I might actually lower bounce_score_threshold from 4.0 to 3.0, but here's what I would expect with your settings:

A user has to bounce on 4 separate days, i.e. 4 messages in your case, with less than 65 days between bounces, at which point the user's bounce score will reach 4 and the user's delivery will be disabled by bounce and if bounce_notify_owner_on_disable is Yes, you will get a notice of the disable with the triggering bounce attached.

This won't happen until you've sent at least 4 messages to the list, i.e. you won't see anything for 2 to 4 months.

When the user's delivery is disabled, she is sent a warning, and then two more at 10 day intervals and then is removed after 10 more days, i.e. 30 days after delivery was disabled. If bounce_notify_owner_on_removal is Yes, you will be notified of this.

Note that the unsubscribes are not directly tied to a recent post. Unsubscribes occur because the user's delivery was disabled 30 days prior.

Bounce scoring and disabling does not necessarily occur immediately after a post. In many cases it will because the recipients MX will not accept the message from the Mailman server, but in other cases, errors may cause delayed delivery which doesn't become fatal the a few days. In other cases, there are several hops to the ultimate delivery, and the bounce DSN doesn't make it back to the Mailman server for up to a few days.

Still, I wouldn't expect a steady incremental removal such as you describe.

Note that things may not be as I describe. The behavior I describe is correct for all Mailman 2.1.x versions except 2.1.5 and 2.1.6+ for sites which have set VERP_PROBES = Yes in mm_cfg.py (Pre 2.1.5 there were no VERP probes, in 2.1.5 VERP_PROBES = Yes was the default, since that release the default is VERP_PROBES = No).

If VERP_PROBES is Yes, when a users bounce score reaches the threshold, the user's score is reset and the user is sent a probe message with a VERPed envelope sender, and only if and when the probe bounces is the user's delivery disabled.

Note also that the sending of all but the first warning to the user and the ultimate unsubscription is under control of Mailman's cron/disabled and won't occur if that cron is not being run.

If you still have questions about this, tell us what Mailman version this is, whether or not you are receiving bounce_notify_owner_on_disable notices (if it's Off, turn it On), how they correlate with the unsubscribes and whether the attached DSNs look like real bounces, and if you have access to Mailman's logs, what's in the 'bounce' log going back as far as you can.

Note that it is possible, though I would think unlikely in the volume you notice, the users don't bounce every message, but bounce often enough to have delivery ultimately disabled.

It occurs to me that a much more likely scenario is you are on a hosted Mailman instance or using a 'smarthost' outgoing MTA, and some fraction of every list post is being bounced because you are running afoul of some recipient limit/rate limit imposed by the host.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan