Hi,
I've been using .forward to forward Email from some user mailboxes to other addresses. Normally this works just fine, but a few weeks ago a situation happened which demonstrates how it can be an epic fail. I had a Mailman/DNS problem after upgrading a lot of packages. A message came in, Mailman couldn't properly look up the DMARC policy of the sending ISP, didn't munge the From: and sent the message on its way, and of course the message was from AOL, just about everybody rejected it, I woke up to fifty-five bounce reports…and all those bounce reports were also forwarded to an Email account on an Internet by telephone service, where deleting them was extremely slow.
What I'm looking for is possibly something that checks mailboxes from time to time, and forwards all incoming messages that meet certain parameters, taking care of DMARC difficulties along the way so the forwarded messages will be accepted by the remote servers. E.G. my mom uses that net by phone service, and would like to see Email which comes to her regular Email address, but doesn't want to spend time deleting Amazon order confirmations, Mailman moderation notices, and other routine, automated, or irrelevant messages. Does such a thing exist?
Thanks for any help,
Jayson
On 11/24/18 10:17 PM, Jayson Smith wrote:
Hi,
Hi,
I've been using .forward to forward Email from some user mailboxes to other addresses. Normally this works just fine, but a few weeks ago a situation happened which demonstrates how it can be an epic fail. I had a Mailman/DNS problem after upgrading a lot of packages. A message came in, Mailman couldn't properly look up the DMARC policy of the sending ISP, didn't munge the From: and sent the message on its way, and of course the message was from AOL, just about everybody rejected it, I woke up to fifty-five bounce reports…and all those bounce reports were also forwarded to an Email account on an Internet by telephone service, where deleting them was extremely slow.
Oy vey.
What I'm looking for is possibly something that checks mailboxes from time to time, and forwards all incoming messages that meet certain parameters, taking care of DMARC difficulties along the way so the forwarded messages will be accepted by the remote servers. E.G. my mom uses that net by phone service, and would like to see Email which comes to her regular Email address, but doesn't want to spend time deleting Amazon order confirmations, Mailman moderation notices, and other routine, automated, or irrelevant messages. Does such a thing exist?
This really sounds like the job of an intelligent Local Delivery Agent. Procmail is (used to be) the quintessential LDA for unix. I think Maildrop is a modern replacement. I suspect that more complex mail stores have similar functionality.
In short, you configure the LDA to intelligently handle various messages and decide what to do with them. You would likely want to filter messages matching one (or more) pattern(s) and then forward other messages matching, possibly with some form of munging.
It may be possible to press Mailman into this service by creating a list of one (or few) subscribers and relying on Mailman's filters / topics to decide who to deliver to. But I feel like this is not what Mailman is designed to do and would likely be fraught with failure.
Thanks for any help,
You're welcome. Good luck.
P.S. Feel free to email me directly / off list if you want help with Procmail.
-- Grant. . . . unix || die
On Sun, 2018-11-25 at 00:17 -0500, Jayson Smith wrote:
I've been using .forward to forward Email from some user mailboxes to other addresses. Normally this works just fine, but a few weeks ago a situation happened which demonstrates how it can be an epic fail. I had a Mailman/DNS problem after upgrading a lot of packages.
How does a .forward problem relate to Mailman? Please excuse my ignorance if this is obvious. I use Courier-MTA and the equivalent facility is the .courier file, which redirects email. In the case of Mailman mailing lists, I use the courier-to-mailman.py as a target in a .courier file to redirect _internally_ into Mailman. Otherwise external redirection in a .forward file (which is an ancient and venerable sendmail facility) can (probably will) cause problems if the redirection is from a "p=reject" domain to a service, such as Gmail, which honors this.
A message came in, Mailman couldn't properly look up the DMARC policy of the sending ISP, didn't munge the From: and sent the message on its way, and of course the message was from AOL, just about everybody rejected it, I woke up to fifty-five bounce reports…and all those bounce reports were also forwarded to an Email account on an Internet by telephone service, where deleting them was extremely slow.
Setting from_is_list to "Munge From" in General Options will apply DMARC mitigation to _all_ From addresses. I believe this takes precedence over dmarc_moderation_action which requires a DNS lookup of the sender's DMARC policy.
It might be a good idea to use this setting preemptively any time you make system changes (or may have problems) which may affect the ability of Mailman to do a proper DNS lookup of a sender.s DMARC policy.
What I'm looking for is possibly something that checks mailboxes from time to time, and forwards all incoming messages that meet certain parameters, taking care of DMARC difficulties along the way so the forwarded messages will be accepted by the remote servers. E.G. my mom uses that net by phone service, and would like to see Email which comes to her regular Email address, but doesn't want to spend time deleting Amazon order confirmations, Mailman moderation notices, and other routine, automated, or irrelevant messages. Does such a thing exist?
Leaving from_is_list set to "Munge From" will take care of any DNS outages, if you don't mind doing this. When it comes to specialized software to do things such as scan mailboxes and take intelligent action, I've found that I pretty much have to write my own, python being my preferred language.
--
Lindsay Haisley | "The first casualty when
FMP Computer Services | war comes is truth."
512-259-1190 |
http://www.fmp.com | -- Hiram W Johnson
On 11/25/18 10:43 AM, Lindsay Haisley wrote:
How does a .forward problem relate to Mailman? Please excuse my ignorance if this is obvious. I use Courier-MTA and the equivalent facility is the .courier file, which redirects email. In the case of Mailman mailing lists, I use the courier-to-mailman.py as a target in a .courier file to redirect _internally_ into Mailman. Otherwise external redirection in a .forward file (which is an ancient and venerable sendmail facility) can (probably will) cause problems if the redirection is from a "p=reject" domain to a service, such as Gmail, which honors this.
A .forward normally does not cause issues with DMARC because .forward redirection normally does not transform the message in ways that break DKIM signatures. Thus if the original message was DKIM signed by a domain "aligned" with the From: domain, the forwarded message should still pass DMARC. There is only an issue if the original sender was relying on SPF only to pass DMARC.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Sun, 2018-11-25 at 11:06 -0800, Mark Sapiro wrote:
A .forward normally does not cause issues with DMARC because .forward redirection normally does not transform the message in ways that break DKIM signatures. Thus if the original message was DKIM signed by a domain "aligned" with the From: domain, the forwarded message should still pass DMARC. There is only an issue if the original sender was relying on SPF only to pass DMARC.
Similarly, if you're using from_is_list in Mailman, or dmarc_moderation_action, the domain name associated with the list must have a proper SPF record.
--
Lindsay Haisley | "The first casualty when
FMP Computer Services | war comes is truth."
512-259-1190 |
http://www.fmp.com | -- Hiram W Johnson
On 11/25/18 11:12 AM, Lindsay Haisley wrote:
Similarly, if you're using from_is_list in Mailman, or dmarc_moderation_action, the domain name associated with the list must have a proper SPF record.
These days, publishing SPF and DKIM signing outgoing mail is good practice in general, but it has nothing to do with DMARC unless the list domain publishes a DMARC policy.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Nov 25, 2018, at 1:06 PM, Mark Sapiro <mark@msapiro.net> wrote:
A .forward normally does not cause issues with DMARC because .forward redirection normally does not transform the message in ways that break DKIM signatures.
Which assumes that the sending system includes a DKIM signature in the original message. If it does not, and the receiving system relies on SPF for DMARC alignment, then DMARC will fail since mail redirected through a .forward will always fail SPF validation.
Sent from my iPhone
On 11/25/18 1:03 PM, Lindsay Haisley (linode) wrote:
mail redirected through a .forward will always fail SPF validation.
That is not always accurate. It is relatively easy to configure an MTA to support Sender Rewriting Scheme, either for everything that is sent out or just things that don't originate from the system.
Thus a .forward is not guaranteed to fail SPF validation. In fact, I would expect SPF validation to succeed on servers that are configured with SRS.
-- Grant. . . . unix || die
On 11/25/18 12:28 PM, Grant Taylor via Mailman-Users wrote:
On 11/25/18 1:03 PM, Lindsay Haisley (linode) wrote:
mail redirected through a .forward will always fail SPF validation.
That is not always accurate. It is relatively easy to configure an MTA to support Sender Rewriting Scheme, either for everything that is sent out or just things that don't originate from the system.
Thus a .forward is not guaranteed to fail SPF validation. In fact, I would expect SPF validation to succeed on servers that are configured with SRS.
Yes, but in the context of this thread which is DMARC, SPF will pass, but the SPF domain won't align with the From: domain so DMARC validation by SPF will fail.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 11/24/18 9:17 PM, Jayson Smith wrote:
I had a Mailman/DNS problem after upgrading a lot of packages. A message came in, Mailman couldn't properly look up the DMARC policy of the sending ISP, didn't munge the From: and sent the message on its way...
What was the lookup issue? I.e., what were the messages in Mailman's error and maybe vette logs? What Mailman version is this?, beginning with Mailman 2.1.25, some failures in DNS lookups of DMARC policy result in mitigations being applied.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro writes:
On 11/24/18 9:17 PM, Jayson Smith wrote:
I had a Mailman/DNS problem after upgrading a lot of packages. A message came in, Mailman couldn't properly look up the DMARC policy of the sending ISP, didn't munge the From: and sent the message on its way...
What was the lookup issue? I.e., what were the messages in Mailman's error and maybe vette logs? What Mailman version is this?, beginning with Mailman 2.1.25, some failures in DNS lookups of DMARC policy result in mitigations being applied.
Another possibility would be to cache the results, as a fallback to the DNS lookup. If the cache hit rate is high enough (as it would be for members-only lists -- the member test would be done first), this should reduce DMARC lookup failures to near zero, which would allow either mitigation-on-failure or quarantine-on-failure strategies by default. A more complex approach would be to lookup in the cache first and trust it until the original lookup expires. Both approaches would have to be opt-in, of course. I don't think either the space impact or performance impact would be very great.
A brief RFE for Mailman 3 (which keeps a much more extensive database, so is more likely to implement) is in https://gitlab.com/mailman/mailman/issues/527.
Steve
-- Associate Professor Division of Policy and Planning Science http://turnbull.sk.tsukuba.ac.jp/ Faculty of Systems and Information Email: turnbull@sk.tsukuba.ac.jp University of Tsukuba Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN
participants (6)
-
Grant Taylor -
Jayson Smith -
Lindsay Haisley -
Lindsay Haisley (linode)
-
Mark Sapiro -
Stephen J. Turnbull