[Mailman-Users] Emails lost due to receipient spam filters

[Stephen J. Turnbull]

Alan Meyer writes:

 > At any rate, my main goal has been to use mailman for an Indian
 > Guides group, which for years has just been an email list of about
 > 50 people.  Mailman seemed like a great option to help ensure that
 > all of the members are accounted for in each email and to have
 > archives available.

The Mailman developer community thanks you for the compliment!

 > The problem that I'm having is that for some people on the list,
 > they don't reliably get their emails.  Mostly, this is from members
 > that are on gmail.com or hotmail.com or in some cases ymail.com.

Members who are sending from Gmail will never receive their own mail
via the list.  Gmail considers it a duplicate of the message in the
Sent folder, and discards the list message.  This is not what you are
reporting, but may account for some of your members' reports.

For the senders you mention above (gmail, hotmail, and ymail), none
has a DMARC policy.  (gmail and ymail have p=none, and hotmail doesn't
have a policy at all.)  For those senders, DMARC is not the reason
your subscribers are losing mail.  The most likely issue that you or
your host can affect is failing to use DKIM and SPF to authenticate
your list.

However, if in ymail.com you are including yahoo.com, yahoo.com *does*
have a p=reject policy, and most users on the large email providers
will either not receive the mail at all (probably hotmail and ymail)
or it will end up in Spam (most of the time for gmail).  Mail sent
from yahoo.com will definitely be discarded or rejected by many
recipient's hosts, and GMail will put it in Spam even if it passes all
their other filters.

 > The "lost" emails get stuck in their spam filter, and in some cases
 > hotmail users don't get their emails at all.
 > 
 > I did some testing with a practice list I created (sending to my
 > own set of yahoo, gmail accounts), where my yahoo account was
 > sending to members on a gmail account.

As mentioned above, yahoo.com != ymail.com.  I recommend that you tell
your Yahoo! users to get another mail account.  Not only is it a PITA
for everybody else to receive their mail, but the reason that Yahoo!
uses p=reject is that they leaked several million users' contact lists
to criminals.  I don't know if they told the affected users that their
data was leaked, but I hope so.

I recommend using Mailman 2.1.18-1, and setting
dmarc_moderation_action to 'Wrap Message'.  This affects only
yahoo.com and aol.com senders's messages, and a variable number of
recipients whose email clients can't handle those messages easily
(this is what Peter Shute is talking about, I can give you some
details if you ask).

 > I found no issue sending directly from yahoo to gmail, but from
 > yahoo->mail list->gmail got caught in the gmail's spam filter.
 > The gmail account spam folder would flag it and give me a note
 > something to the effect of "The sender's yahoo.com address could
 > not be confirmed".

Indeed, that looks like DMARC.

 > Furthermore, in the gmail account, I can mark the email as "Not
 > SPAM" but that doesn't help on subsequent emails.

No, it won't, because yahoo.com has REAL problems with spam appearing
to originate at yahoo.com.  (The spammers have potential for delivery
of millions of spam messages per minute, but they don't waste their
resources on spoofing senders from domains using p=reject).

 > As an experiment, I changed a few settings (from default Yes to
 > No), but these didn't seem to make a difference:
 > - Should messages from this mailing list include the RFC 2369
 >   (i.e. List-*) headers? Yes is highly recommended.
 > - Should postings include the List-Post: header?
 > - Should the Sender header be rewritten for this mailing list to
 >   avoid stray bounces? Yes is recommended.

None of these are relevant to spam filtering AFAIK (but I don't work
for a large email provider so take that with a mineshaftful of salt).

I think the most important steps you can take in general are

(1) Have your MTA sign outgoing mailing list mail with DKIM.

(2) Use SPF.

(3) To mitigate the yahoo/aol p=reject policies, set
    dmarc_moderation_action to 'Wrap Message' (my preference, and
    according to Brian Carpenter's observations, GMail's as well), or
    to 'Munge From' (some people prefer this).

If you don't know what MTA, DKIM, and SPF mean, you probably are using
a hosting service.  If they don't know what MTA, DKIM, and SPF mean,
you should get a new one.  (I like Brian Carpenter because he's often
helpful to other users on this list, but I admit I've never actually
used EMD, the hosting service he runs.  FWIW, I'd say check them out. :-)

If you want to know what all the acronyms stand for and what they are
good for, feel free to ask.  This is just the material I can think of
as fast as I can type it. ;-)  Except I can't say more about the
Yahoo!/AOL contact list issues, just what I already wrote is public
knowledge by now.


Regards,

Steve