Gmail and DKIM problems
A subscriber to one of my lists who posts from gmail has been made aware that some list subscribers do not get his postings because of DKIM setup at gmail. See attached error message.
I understand that he can't do anything about the DKIM setup at gmail.
Can I as list admin do something in the list setup (Mailman 2.29)?
Also, how many subscribers are likely affected by his (or any gmail user's) DKIM setup? That is, are most list subscribers receiving his messages anyway, or is this problem preventing e-mail from him going to most list subscribers?
Thomas Gramstad
----Message was blocked due to DKIM (). From:Anders Ericson *frilanders@gmail.com* <frilanders@gmail.com> To: Biblioteknorge *biblioteknorge@kunnskapsallmenning.no* <biblioteknorge@kunnskapsallmenning.no> Subject: Biblioteket som statussymbol. Lat ungdomen lese, for _born._Vegar,_vatn,_straum_–_og_bibliotek Date:2021-06-24 09:47PM Message ID:1624564030-205058-5323-17319-1 IP:158.36.191.155 (hotell.nuug.no) Envelope From:biblioteknorge-bounces@mailman.kunnskapsallmenning.no Recipients: Recipients Action Reason Delivery Status *kristin.johanne.havstad@arendal.kommune.no* <kristin.johanne.havstad@arendal.kommune.no> Blocked DKIM Not Delivered
Dette kan muligens skyldes at SPF record ikke er helt riktig:
v=spf1 redirect=_spf.google.com
SPF Alignment Domain not found in SPF
Med vennlig hilsen
*Odd Arvid Knudsen* *Teknologiarkitekt* M: +47 488 92 398
*ıkt**.**agder*
*Enklere hverdag*
One thing to try:
Add the following line to /etc/mailman/mm_cfg.py
REMOVE_DKIM_HEADERS = Yes
I did it. It clearly does no harm. Whether it helps, I don't know.
And "probably don't get" may sometimes mean "probably don't check my spam".
I'm thinking about removing all the headers with formail, but I really have no idea why the list mail is so often classified as spam. Nobody tells you how they define spam; it is a secret because they don't want spammers to know about it.
Jon
On 07/01/21 00:37, Thomas Gramstad wrote:
A subscriber to one of my lists who posts from gmail has been made aware that some list subscribers do not get his postings because of DKIM setup at gmail. See attached error message.
I understand that he can't do anything about the DKIM setup at gmail.
Can I as list admin do something in the list setup (Mailman 2.29)?
Also, how many subscribers are likely affected by his (or any gmail user's) DKIM setup? That is, are most list subscribers receiving his messages anyway, or is this problem preventing e-mail from him going to most list subscribers?
Thomas Gramstad
----Message was blocked due to DKIM (). From:Anders Ericson *frilanders@gmail.com* <frilanders@gmail.com> To: Biblioteknorge *biblioteknorge@kunnskapsallmenning.no* <biblioteknorge@kunnskapsallmenning.no> Subject: Biblioteket som statussymbol. Lat ungdomen lese, for _born._Vegar,_vatn,_straum_–_og_bibliotek Date:2021-06-24 09:47PM Message ID:1624564030-205058-5323-17319-1 IP:158.36.191.155 (hotell.nuug.no) Envelope From:biblioteknorge-bounces@mailman.kunnskapsallmenning.no Recipients: Recipients Action Reason Delivery Status *kristin.johanne.havstad@arendal.kommune.no* <kristin.johanne.havstad@arendal.kommune.no> Blocked DKIM Not Delivered
Dette kan muligens skyldes at SPF record ikke er helt riktig:
v=spf1 redirect=_spf.google.com
SPF Alignment Domain not found in SPF
Med vennlig hilsen
*Odd Arvid Knudsen* *Teknologiarkitekt* M: +47 488 92 398
*ıkt**.**agder*
*Enklere hverdag*
Mailman-Users mailing list -- mailman-users@python.org To unsubscribe send an email to mailman-users-leave@python.org https://mail.python.org/mailman3/lists/mailman-users.python.org/ Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/ https://mail.python.org/archives/list/mailman-users@python.org/
-- Jonathan Baron, Professor of Psychology, University of Pennsylvania Home page: https://www.sas.upenn.edu/~baron Editor: Judgment and Decision Making (http://journal.sjdm.org)
On 6/30/21 3:37 PM, Thomas Gramstad wrote:
A subscriber to one of my lists who posts from gmail has been made aware that some list subscribers do not get his postings because of DKIM setup at gmail. See attached error message.
I understand that he can't do anything about the DKIM setup at gmail.
Can I as list admin do something in the list setup (Mailman 2.29)?
Also, how many subscribers are likely affected by his (or any gmail user's) DKIM setup? That is, are most list subscribers receiving his messages anyway, or is this problem preventing e-mail from him going to most list subscribers?
It isn't gmail, it is the recipient MTAs
However, removing the incoming DKIM signatures which Mailman's transformations break as Jon Baron suggest may help.
Also, the message
Dette kan muligens skyldes at SPF record ikke er helt riktig:
v=spf1 redirect=_spf.google.com
SPF Alignment Domain not found in SPF
says the problem is SPF. SPF will always fail on mail forwarded through a list or other forwarding mechanism. You can only avoid this by setting General Options -> from_is_list other than No.
Also you should DKIM sign your outgoing mail and publish your own SPF.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 6/30/21 4:37 PM, Thomas Gramstad wrote:
I understand that he can't do anything about the DKIM setup at gmail.
Nor should he, or anyone else, need to.
Can I as list admin do something in the list setup (Mailman 2.29)?
As others have said, remove incoming DKIM headers from incoming messages, and add your own DKIM headers (signature) to outgoing messages.
This is particularly important if you make /any/ changes to the message as it passes through the mailing list.
Also, how many subscribers are likely affected by his (or any gmail user's) DKIM setup?
It depends on how the sender's domain has configured things; SPF, DKIM, DMARC. Chances are quite good that any sender from a domain using contemporary stringent settings will have problems with any recipient who has a mail server that honors what the sending domain publishes. You have zero control over what the sender's domain does. You have zero control over what the recipient's mail server does. You only have control of what you do with the mailing list.
That is, are most list subscribers receiving his messages anyway, or is
this problem preventing e-mail from him going to most list subscribers?
I'd say the best that you can hope for is for messages from the mailing list to be filed as spam. The worst, which may be more likely, is that the mailing list server develops a bad reputation and ends up blocked by one or more recipient domains.
More sending domains are adopting stringent settings. More receiving servers are honoring stringent settings. It's a multiplicative effect as time goes on. You can either push back or you can update your config. With the multiplicative effect, you will probably need to push back more often.
Stop and think for a moment what's actually happening:
- The sender's mail server is specifying which server(s) are allowed to send email as them and / or apply a cryptographic signature to (part of) the message. They also publish this information so that receiving systems can easily consume it.
- Receiving systems are using the information that senders publish to be able to tell if message are legitimate based on the source and / or cryptographic signature.
So, when you (re)send messages from the mailing list as sending domain (in the SMTP envelope) you are likely running afoul of SPF. When you modify any (signed) part of the message, you are breaking signatures. Thus, recipients see that messages aren't coming from where the sender says they should be and that the cryptographic signature is broken. Hence the receiving server is naturally treating the message from the mailing list as highly suspicious.
To avoid this suspicion:
- Send with your own SMTP envelope address (VERP).
- Use full personalization.
- Remove incoming DKIM signatures.
- Add your own outgoing DKIM signature.
I'd suggest updating your config sooner than later.
-- Grant. . . . unix || die
Grant Taylor via Mailman-Users writes:
Thus, recipients see that messages aren't coming from where the sender says they should be and that the cryptographic signature is broken. Hence the receiving server is naturally treating the message from the mailing list as highly suspicious.
"Naturally" if you don't read RFCs, of course. There are *many* common practices where mail comes from somewhere other than what From says. This is not limited to lists.
Normal folk of good will who hate spam (now, *that* is natural!) will use code by developers who read the RFCs and know that a broken signature has very little information compared to no signature at all, and therefore SHOULD NOT (emphasis in the RFC) treat broken signatures differently from absent ones. Nowadays, you have to be *trying* to break the Internet to treat broken signatures as significant.
To avoid this suspicion:
- Send with your own SMTP envelope address (VERP).
I believe you can change the Return-Path using the list-bounces address setting, but VERP doesn't affect the domain. VERP adds a cookie to the Return-Path which allows identifying the individual recipients whose copies caused bounces. As far as I know, the SMTP MAIL FROM "envelope" address is set by Mailman from the list bounces address. I doubt there's a way to set it to the origin domain in Mailman.
- Use full personalization.
For many recipient domains, this won't help, because they'll identify a mass-mailing by the Message-ID and/or content. However, it may help with some, either because they give spam points because it's not personally addressed, or because they assess that personalization is costly, so spammers won't do it. (Both are bad ideas, but the average user would break the Internet to avoid one spam per day, so there are a lot of domains who do such things ....)
- Remove incoming DKIM signatures.
Removing signatures you expect the list to break (by changing existing headers or the body) is normally harmless. However, if you have any lists that make no changes at all to the existing header or body of the message, but only add new fields to the header, you must not do this for those lists. You'll break all the AOL and Yahoo! users, and possibly unsubscribe half the list.
Also, there are paranoid sites that try to reconstruct the original mail by removing standard header tags and footers and attempting validation. This will break them. (If that applies to your recipients, they'll most likely let you know when you start removing the incoming signatures, so it's not a significant consideration.)
- Add your own outgoing DKIM signature.
This is always a good idea.
I'd suggest updating your config sooner than later.
When everything is an existential threat, nothing is an existential threat. But yeah, if there are delayable tasks, especially related to mailing list admin, these checks and actions should be prioritized:
- outgoing mail is DKIM signed,
- with the right key,
- whose public key is in the right place in DNS, and
- if the lists are coming from a subdomain with a domain-wide key that the DNS record states that subdomains are covered, and
- fixing/adding any of the above that need it, and
- checking that the outgoing MTA's IP/domain is not on any blacklists and remonstrating with the owners if so, and
- adding/configuring/checking a spamcheck on mail incoming to Mailman, and
- if necessary exempting Mailman from any outgoing spamchecks (this doesn't help your mail get through, but the CPU will cheer). Subscribers will appreciate it.
With lower priority, site admins whose lists are mission-critical may want to sign up for various bulk mail certification programs (called various things) at the Internet-scale providers like GMail, Microsoft, and Yahoo!.[1] Those require substantial effort, though. There are other ideas in the FAQ.
Steve
Footnotes: [1] Unfortunately Yahoo! and AOL are now run by Verizon, which seems to want to break the Internet, or at least Internet mail, so I suspect their programs are losing usefulness rapidly.
participants (5)
-
Grant Taylor -
Jon Baron -
Mark Sapiro -
Stephen J. Turnbull -
Thomas Gramstad