On 1/8/14, 10:03 AM, Doug Hutson wrote:

All the people that have e-mailed us about this issue were specifically yahoo addresses. Reports of 4-50 duplicates per user were sent to them. We only have one list of users we send users to, and this is the first time this has ever happened.

The few times that I ran in to this type of problem in the past were not Mailman issues at all. Rather they were symptoms of underlying SMTP issues. My clue when I fought this battle before was that the messages IDs down stream of my server(s) were different in each ""copy of the message. When investigating in my SMTP logs, I found that the delivery attempts were failing in some manner and my SMTP server was considering the delivery a failure and would retry until it saw the expected response. However, the receiving server did considered the delivery to be successful, and as such treated the message(s) as valid. Ultimately, causing duplicated messages.

When investigating the SMTP issue, I found that the problem was related to network connectivity issues, specifically saturated connection on the receiving end. - Once I addressed (worked around) the connectivity issues, the duplicate messages cleared up immediately.

On Jan 8, 2014, at 6:03 AM, Doug Hutson doug@mrdirectint.com wrote:

We've been using mailman for quite a long time now, and suddenly the last newsletter we sent out sent duplicates to random users as far as I can tell.

All the people that have e-mailed us about this issue were specifically yahoo addresses. Reports of 4-50 duplicates per user were sent to them. We only have one list of users we send users to, and this is the first time this has ever happened.

So, my question specifically... does anybody have advice in what I should look for as to why this happened?

I would start with looking at the logs of your mail server. You want to figure out where the duplication is occurring. Was it sent to Yahoo more than once? Or was it accepted by Yahoo just once? If the latter, the duplication is happening at Yahoo and it is not your problem to solve.

You didn’t say what mail server is being used but SMTP is designed not to lose mail. Therefore, a duplicate is preferable to losing mail. If the receiving SMTP server receives the mail but does not tell the sending server it has it (sending server sends “end of message” but receiving server does not acknowledge), then the sending server assumes something went wrong and tries to send it again.

My guess would be it’s either happening between your outgoing SMTP server and Yahoo (or the next server if you’re relaying through another server) or internally in Yahoo. It’s very unlikely, particularly since you say it only happens with Yahoo, that it’s a problem in Mailman. Rather, it’s happening after Mailman is done with the message so is not a Mailman problem per se.