The internal documentation in the admin screens for 2.1.18 is a bit confusing with regard to Reply-To munging.
In the doc for the from_is_list option we have "It replaces the poster's address in the From: header with the list address and adds the poster to the Reply-To: header, but the anonymous_list and Reply-To: header munging settings below take priority."
Which Reply-To: header munging settings take priority, and how should one set them so that they don't override from_is_list?
If dmarc_moderation_action overrides from_is_list, as the doc for it says it does, is it also overridden by Reply-To: header munging settings?
Also, in the doc for reply_goes_to_list, we see "When set to Poster [the default], no Reply-To: header is added by Mailman". Does this mean that this overrides from_is_list, which if set says that it causes the original From header to be inserted into the Reply-To header?
I can probably figure this out, but it might be good to explain this a bit more completely in the Mailman internal docs. It's not really clear exactly how these options relate, and how the precedence of settings is organized. These may be stupid questions, but I can just about guarantee you that all my list admins will trip on them :(
-- Lindsay Haisley | "Everything works if you let it" FMP Computer Services | 512-259-1190 | --- The Roadie http://www.fmp.com |
On 04/30/2014 12:56 PM, Lindsay Haisley wrote:
The internal documentation in the admin screens for 2.1.18 is a bit confusing with regard to Reply-To munging.
In the doc for the from_is_list option we have "It replaces the poster's address in the From: header with the list address and adds the poster to the Reply-To: header, but the anonymous_list and Reply-To: header munging settings below take priority."
Which Reply-To: header munging settings take priority, and how should one set them so that they don't override from_is_list?
If you set anonymous_list = Yes the post will be fully anonymized as before which includes setting the From: to the list address. Thus, setting anonymous_list = Yes obviates the need for From: header munging because you're already doing it.
However, the statment "the anonymous_list and Reply-To: header munging settings below take priority." is probably not correct. It comes from 2.1.16 which was a bit different with respect to Reply-To:.
In 2.1.18, with anonymous_list - Yes and from_is_list = No, From: will be "List's description <list posting address>" and there will be no Reply-To:. If from_is list is Munge (or Wrap) the (outer) From: will be "List's description via list's real_name <list posting address>" and the (outer) Reply-To will be "List's description <list posting address>"
So the from_is_list options add 'via real_name' to the From: and add a redundant Reply-To: the list.
from_is_list interacts with first_strip_reply_to and reply_goes_to_list in the way one intuitively expects. I.e. first_strip_reply_to and reply_goes_to_list behave as always and from_is_list just adds the posters From: to Reply-To: if it isn't there already (and in the case of an anonymous list, the poster's From: has already been set to the list.
That text should probably be changed. I don't like to change strings that have already been translated and this one was for 2.1.16 for a few languages, but I should fix it.
If dmarc_moderation_action overrides from_is_list, as the doc for it says it does, is it also overridden by Reply-To: header munging settings?
As I explained above, from_is_list works in all cases. And, the same is true for dmarc_moderation_action if it applies.
Also, in the doc for reply_goes_to_list, we see "When set to Poster [the default], no Reply-To: header is added by Mailman". Does this mean that this overrides from_is_list, which if set says that it causes the original From header to be inserted into the Reply-To header?
No. The poster's address will be added to Reply-To: in all cases where from_is_list or dmarc_moderation_action rewriting or wrapping is done.
I can probably figure this out, but it might be good to explain this a bit more completely in the Mailman internal docs. It's not really clear exactly how these options relate, and how the precedence of settings is organized. These may be stupid questions, but I can just about guarantee you that all my list admins will trip on them :(
I hear you. It is badly explained and I need to fix it. Thanks for raising this.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 04/30/2014 03:18 PM, Mark Sapiro wrote:
On 04/30/2014 12:56 PM, Lindsay Haisley wrote:
I can probably figure this out, but it might be good to explain this a bit more completely in the Mailman internal docs. It's not really clear exactly how these options relate, and how the precedence of settings is organized. These may be stupid questions, but I can just about guarantee you that all my list admins will trip on them :(
I hear you. It is badly explained and I need to fix it. Thanks for raising this.
I changed it. In the 2.1.18 final it will say:
from_is_list (general): Replace the sender with the list address to conform with policies like DMARC.
Replace the sender with the list address to conform with policies like ADSP and DMARC. It replaces the poster's address in the From: header with the list address and adds the poster to the Reply-To: header.
If this is set to Wrap Message, just wrap the message in an outer message From: the list with Content-Type: message/rfc822.
These settings play as expected with the anonymous_list and Reply-To: header munging settings below with the exception of adding "via real_name" to the display name in the From: for an anonymous list and adding the poster's address to Reply-To: in almost all cases.
If anonymous_list is Yes, there is no reason to set from_is_list to anything other than No.
If dmarc_moderation_action applies to this message with an action other than Accept, that action rather than this is applied
Note I also removed the bit about SPF and DKIM signing. They actually may help with acceptance of your list mail by some ESPs, but not because of DMARC, and the note could discourage people from using this when it shouldn't.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Wed, 2014-04-30 at 15:18 -0700, Mark Sapiro wrote:
I hear you. It is badly explained and I need to fix it. Thanks for raising this.
Mailman is open source software and I use it. It's my job :)
-- Lindsay Haisley | "Everything works if you let it" FMP Computer Services | 512-259-1190 | --- The Roadie http://www.fmp.com |
Mark Sapiro writes:
I changed it. In the 2.1.18 final it will say:
from_is_list (general): Replace the sender with the list address to conform with policies like DMARC.
Replace the sender with the list address to conform with policies like ADSP and DMARC. It replaces the poster's address in the From: header with the list address and adds the poster to the Reply-To: header.
If this is set to Wrap Message, just wrap the message in an outer message From: the list with Content-Type: message/rfc822.
Ouch! I'd really like to change this variable's name! (It's not very easy to understand anyway; grammatically the semantics are "whatever you see in From is {a, the} list", not "replace whatever is in From with the list's address".)
Something like "from_alignment", with values None (don't try to align domains in From), 'shift author' (From into Reply-To, <list> into From), or 'wrap message'.
May as well rewrite the doc ... here goes:
from_alignment: Try to ensure that From is not "misaligned" with
the author's domain, to conform with protocols like DMARC.
[FIXME: I don't see how to avoid the double negative. Help?!]
This setting replaces the from_is_list setting, which is now
deprecated. Existing from_is_list settings will be respected.
Several protocols now in wide use attempt to ensure that use of
the domain in the author's address (ie, in the From header field)
is authorized by that domain. These protocols may be incompatible
with common list features such as footers, causing participating
email services to bounce list traffic merely because of the
address in the From field. *This has resulted in members being
unsubscribed despite being perfectly able to receive mail.*
The following actions are applied to messages where use of such a
protocol is detected by Mailman. [FIXME: Is that correct?]
Valid values:
'no': Do nothing special. This is appropriate for anonymous lists.
It is appropriate for dedicated announcement lists, unless the
"From" address is not within the Mailman host's domain. [FIXME:
Maybe None is a better value here. Of course that's not backward
compatible, but with the name change it would be possible to check
the old from_is_list.]
'shift author': Shift the address(es) in From to Reply-To
(preserving existing addresses in Reply-To), and insert the list's
[posting?] address in From.
'wrap message': Treat the message as a MIME forward with list in
From and the original message encapsulated in a MIME message/rfc822
part. Subscribers will perceive this as a "one message digest".
[FIXME: Should this respect the MIME vs. legacy encapsulation
('digest') setting? If 'yes', that setting should move to General
or so?]> These settings play as expected with the anonymous_list and Reply-To:
What does "as expected" mean? (If *I* have to ask.... :-)
header munging settings below with the exception of adding "via real_name" to the display name in the From: for an anonymous list and
?? Adding real name to From in an *anonymous* list?
adding the poster's address to Reply-To: in almost all cases.
If anonymous_list is Yes, there is no reason to set from_is_list to anything other than No.
Unnecessary with my wording above.
If dmarc_moderation_action applies to this message with an action other than Accept, that action rather than this is applied
This doesn't seem correct. True, if Reject (aka "emit backscatter") or Discard, the message will never reach this point. But if it's Hold, this processing will be applied if the message is accepted by the moderator. How about
See also dmarc_moderation_action (which will be applied earlier in
processing than this feature).
On Apr 30, 2014, at 9:57 PM, Stephen J. Turnbull stephen@xemacs.org wrote:
May as well rewrite the doc ... here goes:
from_alignment: Try to ensure that From is not "misaligned" with the author's domain, to conform with protocols like DMARC. [FIXME: I don't see how to avoid the double negative. Help?!]
Seems to me saying “Try to ensure that 'From:' is “aligned” with …” does it. I’d prefer to put the header field name in quotes or otherwise distinguish it. Otherwise, it can be difficult to parse - is “from” the header or a preposition.
But, I don’t like “author’s domain”. Who or what is the author? Why not just say “the Mailman server’s domain” since that’s what it’s going to be aligned with.
'no': Do nothing special. This is appropriate for anonymous lists. It is appropriate for dedicated announcement lists, unless the "From" address is not within the Mailman host's domain. [FIXME: Maybe None is a better value here. Of course that's not backward compatible, but with the name change it would be possible to check the old from_is_list.]
None is better. No would only be appropriate if ‘yes’ was the other option. But backwards compatibility is important too (even if it’s not to most large computer companies :-( ).
-- Larry Stone lstone19@stonejongleux.com http://www.stonejongleux.com/
On Wed, 2014-04-30 at 17:42 -0700, Mark Sapiro wrote:
If this is set to Wrap Message, just wrap the message in an outer message From: the list with Content-Type: message/rfc822.
Since this is the _outer_ wrapper, shouldn't this be multipart/mixed? The inner _real_ list post is Content-Type: message/rfc822.
-- Lindsay Haisley | "Everything works if you let it" FMP Computer Services | 512-259-1190 | --- The Roadie http://www.fmp.com |
On Wed, 2014-04-30 at 17:42 -0700, Mark Sapiro wrote:
Note I also removed the bit about SPF and DKIM signing. They actually may help with acceptance of your list mail by some ESPs, but not because of DMARC, and the note could discourage people from using this when it shouldn't.
DKIM signing isn't readily available for many MTAs, so this is good. My server uses courier-MTA for which DKIM signing isn't well developed.
-- Lindsay Haisley | "Everything works if you let it" FMP Computer Services | 512-259-1190 | --- The Roadie http://www.fmp.com |
On 04/30/2014 07:57 PM, Stephen J. Turnbull wrote:
May as well rewrite the doc ... here goes:
from_alignment: Try to ensure that From is not "misaligned" with the author's domain, to conform with protocols like DMARC. [FIXME: I don't see how to avoid the double negative. Help?!]
I'm not sure what to change at this point. I really don't want another change in the attribute name, but maybe.
I'm also not sure about alignment as that is a technical term in the DMARC spec and may be more technical than we want here.
If I understand the double negative remark, what about 'From is "aligned" ...', but that really is not the issue. The issue is that if the From: domain publishes a DMARC p=reject policy, that domain must align with that of a valid DKIM signature or a valid SPF envelope sender/server. The SPF won't align because the envelope is from the list's domain for bounce processing, and the DKIM sig from the author's domain won't validate because of list transformations on the message.
We are really rewriting the From: header (I need to remove the 'sender' wording) to avoid it's containing the author's address because that address won't pass DMARC.
This setting replaces the from_is_list setting, which is now deprecated. Existing from_is_list settings will be respected. Several protocols now in wide use attempt to ensure that use of the domain in the author's address (ie, in the From header field) is authorized by that domain. These protocols may be incompatible with common list features such as footers, causing participating email services to bounce list traffic merely because of the address in the From field. *This has resulted in members being unsubscribed despite being perfectly able to receive mail.* The following actions are applied to messages where use of such a protocol is detected by Mailman. [FIXME: Is that correct?]
The current from_is_list applies to all list messages. dmarc_moderation_action applies only to messages From: a domain "where use of such a protocol is detected by Mailman."
Valid values: 'no': Do nothing special. This is appropriate for anonymous lists. It is appropriate for dedicated announcement lists, unless the "From" address is not within the Mailman host's domain. [FIXME: Maybe None is a better value here. Of course that's not backward compatible, but with the name change it would be possible to check the old from_is_list.] 'shift author': Shift the address(es) in From to Reply-To (preserving existing addresses in Reply-To), and insert the list's [posting?] address in From. 'wrap message': Treat the message as a MIME forward with list in From and the original message encapsulated in a MIME message/rfc822 part. Subscribers will perceive this as a "one message digest". [FIXME: Should this respect the MIME vs. legacy encapsulation ('digest') setting? If 'yes', that setting should move to General or so?]
I don't want to go the FIXME route. It's too hard for this release. Also, are you suggesting doing this for all messages based on what is now Digest options-> mime_is_default_digest or doing it per user based on the user's "Get MIME or Plain Text Digests?" (which has a value for everyone even if they don't get digests). Of course, we are only concerned with non-digest members here and their value is probably the list default anyway.
Also, this (legacy encapsulation) really only differs from the Munge
From option in that a few headers are copied to the body of the message and non-text/plain part are scrubbed, and I don't know how valuable it would be.
These settings play as expected with the anonymous_list and Reply-To:
What does "as expected" mean? (If *I* have to ask.... :-)
Point taken.
header munging settings below with the exception of adding "via real_name" to the display name in the From: for an anonymous list and
?? Adding real name to From in an *anonymous* list?
real_name refers the the list attribute which is the list name with possibly different capitalization, but I see it should be changed.
adding the poster's address to Reply-To: in almost all cases.
If anonymous_list is Yes, there is no reason to set from_is_list to anything other than No.
Unnecessary with my wording above.
If dmarc_moderation_action applies to this message with an action other than Accept, that action rather than this is applied
This doesn't seem correct. True, if Reject (aka "emit backscatter") or Discard, the message will never reach this point. But if it's Hold, this processing will be applied if the message is accepted by the moderator. How about
Hold is not an option for dmarc_moderation_action. it is the action which applies to messages From: a domain with DMARC policy p=reject an optionally p=quarantine. The possible actions are Accept, Munge From, Wrap Message, Reject or Discard
See also dmarc_moderation_action (which will be applied earlier in processing than this feature).
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Thu, 2014-05-01 at 11:57 +0900, Stephen J. Turnbull wrote:
from_alignment: Try to ensure that From is not "misaligned" with the author's domain, to conform with protocols like DMARC. [FIXME: I don't see how to avoid the double negative. Help?!]
from_alignment: Try to ensure that From is "aligned" with
the author's domain, to conform with protocols like DMARC.-- Lindsay Haisley | "Everything works if you let it" FMP Computer Services | 512-259-1190 | --- The Roadie http://www.fmp.com |
On 04/30/2014 09:06 PM, Lindsay Haisley wrote:
On Wed, 2014-04-30 at 17:42 -0700, Mark Sapiro wrote:
If this is set to Wrap Message, just wrap the message in an outer message From: the list with Content-Type: message/rfc822.
Since this is the _outer_ wrapper, shouldn't this be multipart/mixed? The inner _real_ list post is Content-Type: message/rfc822.
Actually, initially, an outer message with Content-Type: message/rfc822 is created with body equal to the original message. I.e. as described.
But, by the time the user receives it, it will have been decorated with msg_header and/or msg_footer assuming at least one is non empty, and it then becomes some subset of
multipart/mixed text/plain <- the msg_header message/rfc822 <- the original message text/plain <- msg_footer
but initially, it is just a single part message/rfc822 message containing the original message, analogous to a single part text/plain message containing a plain text body.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Larry Stone writes:
Seems to me saying “Try to ensure that 'From:' is “aligned” with …” does it.
No. The problem is the author's email provider (ie, the mail domain of the person whose address is in the original From). For most lists, Mailman does *not* want "From" to be aligned with any particular domain, it wants to just pass "From" through. RFC 5322 quite clear that this is the correct behavior (as are all of its predecessors).
I’d prefer to put the header field name in quotes or otherwise distinguish it. Otherwise, it can be difficult to parse - is “from” the header or a preposition.
Sure.
But, I don’t like “author’s domain”. Who or what is the author?
"The author" is the person responsible for the content of the message, per RFC 5322 (and all of its predecessors). You could substitute "the original 'From' address" if that seems more intelligible to non-technical admins.
Why not just say “the Mailman server’s domain” since that’s what it’s going to be aligned with.
That may or may not be true, depending on how the host is set up. For example, it may not participate in SPF or DKIM at all. DMARC alignment is a rather complex concept. I do not think it's a good idea to have sloppy wording here.
Mark Sapiro writes:
I'm not sure what to change at this point. I really don't want another change in the attribute name, but maybe.
Yeah, I know. On the other hand, now that it really matters, this is probably the last chance to make such a change.
I'm also not sure about alignment as that is a technical term in the DMARC spec and may be more technical than we want here.
Sure. But "from rewriting" is something we do for other reasons (anonymous lists), and saying that message/rfc822 encapsulation is "from rewriting" seems way too inaccurate to me.
[FIXME: Should this respect the MIME vs. legacy encapsulation ('digest') setting? If 'yes', that setting should move to General or so?]I don't want to go the FIXME route. It's too hard for this release.
OK.
Also, are you suggesting doing this for all messages based on what is now Digest options-> mime_is_default_digest or doing it per user based on the user's "Get MIME or Plain Text Digests?"
Per user, because of the issues we've heard about specific MUAs having trouble with MIME encapsulation.
Also, this (legacy encapsulation) really only differs from the Munge
From option in that a few headers are copied to the body of the message and non-text/plain part are scrubbed, and I don't know how valuable it would be.
True. I mention it because we've had PRs about MIME encapsulation already.
header munging settings below with the exception of adding "via real_name" to the display name in the From: for an anonymous list and
?? Adding real name to From in an *anonymous* list?
real_name refers the the list attribute which is the list name with possibly different capitalization, but I see it should be changed.
OIC. I don't think you need to mention it here; Mailman should just DTRT. If it's an anonymous list, the list owner should configure 'From' correctly, that's all.
Hold is not an option for dmarc_moderation_action. it is the action which applies to messages From: a domain with DMARC policy p=reject an optionally p=quarantine. The possible actions are Accept, Munge From, Wrap Message, Reject or Discard
I don't understand why we need both this and list_is_from? The latter is a clear violation of RFC 5322, acceptable only because it's one of the approaches the DMARC proponents (and Yahoo!) suggest for mailing lists faced with a DMARC DoS attack. Why not just deprecate list_is_from in favor of dmarc_moderation_action?
Here's what I've got. I didn't change the name of the setting, but I changed its description and all the detail. I now have
from_is_list (general): Replace the From: header address with the list's posting address to mitigate issues stemming from the original From: domain's DMARC or similar policies.
Several protocols now in wide use attempt to ensure that use of the domain in the author's address (ie, in the From: header field) is authorized by that domain. These protocols may be incompatible with common list features such as footers, causing participating email services to bounce list traffic merely because of the address in the From: field. *This has resulted in members being unsubscribed despite being perfectly able to receive mail.*
The following actions are applied to all list messages when selected here. To apply these actions only to messages where the domain in the From: header is determined to use such a protocol, see the dmarc_moderation_action settings under Privacy options... -> Sender filters.
Settings:
No Do nothing special. This is appropriate for anonymous lists. It is appropriate for dedicated announcement lists, unless the From: address of authorized posters might be in a domain with a DMARC or similar policy. It is also appropriate if you choose to use dmarc_moderation_action other than Accept for this list. Munge From This action replaces the poster's address in the From: header with the list's posting address and adds the poster's address to the Reply-To: header. Wrap Message Just wrap the message in an outer message with the From: header containing the list's posting address and with the original From: address added to the Reply-To: header and with Content-Type: message/rfc822. This is effectively a one message MIME format digest.
The transformations for anonymous_list are applied before any of these actions, so if actions other than No are applied on an anonymous list, they will apply to the anonymized message.
The Reply-To: header munging actions below interact with these actions as follows:
first_strip_reply_to = Yes will remove all the incoming Reply-To: addresses but will still add the poster's address to Reply-To: for all three settings of reply_goes_to_list which respectively will result in just the poster's address, the poster's address and the list posting address or the poster's address and the explicit reply_to_address in the outgoing Reply-To: header.
[Note: is the above what we want? I think so, but others are adding a header something like X-Mailman-Originally-From: (see the "The future options for mailing list managers" section at http://onlinegroups.net/blog/2014/05/01/dmarc-taking-responsibility-sending-...)]
These actions do not apply to messages in digests or archives or sent to usenet via the Mail<->News gateways.
If dmarc_moderation_action applies to this message with an action other than Accept, that action rather than this is applied
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 04/30/2014 11:58 PM, Stephen J. Turnbull wrote:
Why not just deprecate list_is_from in favor of dmarc_moderation_action?
I don't think I can for two reasons. One is technical. dmarc_moderation_action is unreliable. If the DNS lookup times out, the message is assumed unaffected by DMARC. This could be reversed, and in any case, I plan to make the timeouts an mm_cfg.py setting, but I think this is an issue. It also requires installation of the 3rd party dnspython package, and if that's not there, all messages pass. I did build a test into configure so it will error out if dnspython isn't installed. None of these is a show stopper for this feature, but some may just want to apply whatever action to all messages.
The other is social. My largest and highest traffic production list is the discussion list for my bicycle club.
If I could, I would set dmarc_moderation_action to Reject with a gentle suggestion to find another ESP to post from, but I can't. One particular member of the board of directors is a very vocal and not very technical Yahoo user who I think would have a fit if her mail were "singled out" for differing treatment, even if only that hers was munged and mine wasn't. I have tried in the past to programmatically hold or reject "me too" posts and others that have way more quoted that original material. (Almost everyone top posts and quotes the entire message being replied to.) I was forced by a few vociferous complainers who got the boards attention to back off.
Anyway, my point is list owners don't always have the freedom to do whatever they want. If their list serves an organization, the policies of that organization can override good sense.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Thu, 2014-05-01 at 20:29 -0700, Mark Sapiro wrote:
first_strip_reply_to = Yes will remove all the incoming Reply-To: addresses but will still add the poster's address to Reply-To: for all three settings of reply_goes_to_list which respectively will result in just the poster's address, the poster's address and the list posting address or the poster's address and the explicit reply_to_address in the outgoing Reply-To: header.
If first_strip_reply_to = No there are two possible situations which aren't covered in your (much improved!) self-doc. Either the original poster included a Reply-To:, or not. If not, then I assume the original From: address is put into the Reply-To: address. Yes? If the original poster included a Reply-To: address then DMARC forces us into a situation in which we can't avoid information loss. Either the poster's Reply-To: is overwritten with the original From:, or the original Reply-To: is retained and the original From: address is lost. Which is it?
-- Lindsay Haisley | "Everything works if you let it" FMP Computer Services | 512-259-1190 | --- The Roadie http://www.fmp.com |
Mark Sapiro writes:
from_is_list (general): Replace the From: header address with the list's posting address to mitigate issues stemming from the original From: domain's DMARC or similar policies.
That's good!
[snip my suggestion :]
The following actions are applied to all list messages when selected here. To apply these actions only to messages where the domain in the From: header is determined to use such a protocol, see the dmarc_moderation_action settings under Privacy options... -> Sender filters.
Good! Maybe even "encourage" use of d_m_a?
No Do nothing special. This is appropriate for anonymous lists.
[snip]
The transformations for anonymous_list are applied before any of these actions, so if actions other than No are applied on an anonymous list, they will apply to the anonymized message.
This may be confusing?
The Reply-To: header munging actions below interact with these actions as follows:
first_strip_reply_to = Yes will remove all the incoming Reply-To: addresses but will still add the poster's address to Reply-To: for all three settings of reply_goes_to_list which respectively will result in just the poster's address, the poster's address and the list posting address or the poster's address and the explicit reply_to_address in the outgoing Reply-To: header.
[Note: is the above what we want? I think so, but others are adding a header something like X-Mailman-Originally-From:
IIUC, yes, that's what we want. OnlineGroups has some features Mailman doesn't (yet?) to handle the "reply munging" issue AIUI.
(1) X- fields are deprecated. They don't actually help in creating "private" protocols, and (not relevant to us here, I think) they make it difficult to upgrade to the standardized version.
(2) I think this is pretty useless (with one exception), because most MUAs won't display the information. Even with "Show Source" (how many AOLers use that?), you have to dig through a thicket of trace fields and spam scores. Better to log the information. But why do that when we archive the original message as received? (In fact, if we do this correctly it would be possible to post-archive DKIM-verify messages! GSoC 2015? :-)
(see the "The future options for mailing list managers" section at http://onlinegroups.net/blog/2014/05/01/dmarc-taking-responsibility-sending-...)]
They don't seem to think it's terribly useful though, it's just a sort of trace field. BTW, that blog also says
The attackers succeeded in accessing about 20% of AOL users’ email
accounts and obtaining details of their contacts.I hope that means that AOL is now down to the 100 Stupidest On-Line Americans, of whom 20 were fooled.... But I digress.
These actions do not apply to messages in digests or archives or sent to usenet via the Mail<->News gateways.
Is that true of dmarc_moderation_action, too? (I assume so and would consider it a bug if not.)
Mark Sapiro writes:
dmarc_moderation_action is unreliable. If the DNS lookup times out, the message is assumed unaffected by DMARC.
Ouch. I suppose you could hard-code a list of miscreants, er, domains that have used p=reject and fall back on that (including a check for a change in policy of DMARC DoS'ers that results in an email to owner).
If I could, I would set dmarc_moderation_action to Reject with a gentle suggestion to find another ESP to post from, but I can't.
Heck, even I don't recommend that (I do it, but that's only because I *can* -- as I've mentioned my users are all looking for excuses to switch to GMail anyway if they haven't done so already).
One particular member of the board of directors is a very vocal and not very technical Yahoo user who I think would have a fit if her mail were "singled out" for differing treatment, even if only that hers was munged and mine wasn't.
Tell her it's mandated by Yahoo! and hard-coded in Mailman (blame Barry! and don't tell her about the Time Machine), *your* hands are tied. :-)
I have tried in the past to programmatically hold or reject "me too" posts and others that have way more quoted that original material. (Almost everyone top posts and quotes the entire message being replied to.)
But this is different: it really is censorship. It's censorship I agree with, it's censorship that doesn't actually infringe on freedom of expression, but it is prohibiting certain (obnoxious) forms of expression.
N.B. If top posting bothered me (in channels where it is customary, it doesn't bother me any more :-), I would implement a Handler that removes trailing quoted material and substitutes a link to the archives (if possible to the In-Reply-To message).
On Thu, May 01, 2014 at 08:29:30PM -0700, Mark Sapiro wrote:
Here's what I've got. I didn't change the name of the setting, but I changed its description and all the detail. I now have
Do you have a setting to change From: user@domain to From: user@domain.INVALID
- that is the hack I would like to use.
Andrew Partan
On 05/01/2014 09:26 PM, Lindsay Haisley wrote:
If first_strip_reply_to = No there are two possible situations which aren't covered in your (much improved!) self-doc. Either the original poster included a Reply-To:, or not. If not, then I assume the original From: address is put into the Reply-To: address. Yes?
Yes.
If the original poster included a Reply-To: address then DMARC forces us into a situation in which we can't avoid information loss. Either the poster's Reply-To: is overwritten with the original From:, or the original Reply-To: is retained and the original From: address is lost. Which is it?
Neither. The poster's From: address is merged with the other addresses in the original Reply-To:. I.e. The Poster's From: address will be there as will all the other addresses in the original Reply-To:, but there will be no duplicates.
What do I need to change to make this clear. Note that earlier under the Munge From action, it says "... and adds the poster's address to the Reply-To: header" and under the Wrap Message action it says "with the original From: address added to the Reply-To: header".
So it seems clear to me that we're *adding* the From: address to Reply-To: and the only question is how does first_strip_reply_to affect this, and the answer is if it's Yes, the Reply-To: we're adding to was stripped and is empty, and if No we're adding to the original. Do I have to repeat that last bit further down?
Maybe instead of "the Reply-To: header" in the two action statements, I should say "the addresses in the original Reply-To: header". I.e. "... and adds the poster's address to the addresses in the original Reply-To: header" and "with the original From: address added to the addresses in the original Reply-To: header". Does that help?
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 05/01/2014 09:33 PM, Stephen J. Turnbull wrote:
Mark Sapiro writes:
The transformations for anonymous_list are applied before any of these actions, so if actions other than No are applied on an anonymous list, they will apply to the anonymized message.
This may be confusing?
How about "... if actions other than No are applied on an anonymous list, they will be redundant."
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 05/01/2014 09:57 PM, Andrew Partan wrote:
Do you have a setting to change From: user@domain to From: user@domain.INVALID
- that is the hack I would like to use.
No, not currently. It is an interesting idea, but it may cause issues in delivery of mail From: a non-existent domain.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Thu, 2014-05-01 at 22:09 -0700, Mark Sapiro wrote:
So it seems clear to me that we're *adding* the From: address to Reply-To: and the only question is how does first_strip_reply_to affect this, and the answer is if it's Yes, the Reply-To: we're adding to was stripped and is empty, and if No we're adding to the original. Do I have to repeat that last bit further down?
I hadn't considered that a Reply-To: address can be plural, which makes perfect sense. A single sentence, perhaps just a reference to other text, covering first_strip_reply_to = No might be in order to pair with your explicit discussion of first_strip_reply_to = Yes.
The whole issue is complex, and the measures in Mailman to address it are similarly complex. Your changes to the internal docs are certainly an improvement and probably about as good as can be done with a bad situation. There may be a problem with being _too_ wordy in explaining it.
Here's a suggestion:
first_strip_reply_to = Yes will remove all the incoming
Reply-To:
addresses but will still add the poster's address to Reply-To:
for all three settings of reply_goes_to_list which respectively
will result in just the poster's address, the poster's address
and the list posting address or the poster's address and the
explicit reply_to_address in the outgoing Reply-To: header. If
first_strip_reply_to = No the poster's address in the From:
header, if not already included in the Reply-To:, will be
appended to any existing Reply-To: address(es).
Last sentence added. Is this correct, and reasonable?
-- Lindsay Haisley | "Everything works if you let it" FMP Computer Services | 512-259-1190 | --- The Roadie http://www.fmp.com |
Andrew Partan writes:
Do you have a setting to change From: user@domain to From: user@domain.INVALID - that is the hack I would like to use.
Seems reasonable, but for the reason Mark gave and because it makes personal replies a little bit harder, I *personally* would tend to avoid it, and I recommend being careful to observe what's happening if you try it (watch logs, listen for user complaints, etc).
Mark Sapiro writes:
On 05/01/2014 09:33 PM, Stephen J. Turnbull wrote:
Mark Sapiro writes:
The transformations for anonymous_list are applied before any of these actions, so if actions other than No are applied on an anonymous list, they will apply to the anonymized message.
This may be confusing?
How about "... if actions other than No are applied on an anonymous list, they will be redundant."
You already said that above, so it won't hurt to say it here too. But what I'm worried about (which was not clear, sorry!) is that if somebody *does* turn one of these options on, the behavior they observe will be confusing. Ie, I wonder if
It is probably *not* useful to apply these options to an anonymous
list, and if you do need to do so, the result may be surprising.isn't the most accurate statement of affairs.
Steve
(Despite the subject line, this follows up a digression by correcting some mistaken information about an e-mail attack on AOL.)
On 5/2/2014 12:33 AM, Stephen J. Turnbull wrote:
BTW, that blog [http://onlinegroups.net/blog/2014/05/01/dmarc-taking-responsibility-sending-...] also says
The attackers succeeded in accessing about 20% of AOL users’ email accounts and obtaining details of their contacts.I hope that means that AOL is now down to the 100 Stupidest On-Line Americans, of whom 20 were fooled.... But I digress.
What it actually means is that onlinegroups.net miscopied the percentage from the AOL blog! The figure given by AOL is 2%. (Discovered because I'm a compulsive looker-up of sources....)
-- Larry Kuenning larry@qhpress.org
On Thu, 01 May 2014 at 22:52:27 -0700, Mark Sapiro wrote:
Do you have a setting to change From: user@domain to From: user@domain.INVALID - that is the hack I would like to use.
No, not currently. It is an interesting idea, but it may cause issues in delivery of mail From: a non-existent domain.
None of the current options to try to work around the DMARC breakage work well; all fail in various ways.
Until people figure out real ways of making DMARC work with forwrders & mailing lists (see ietf-822@ietf.org for one place discussions are going on), I think it useful to have more work-around hacks out there so that people can experiment with them to see which ones more-or-less work in different situations.
Andrew Partan
On 05/01/2014 11:29 PM, Lindsay Haisley wrote:
There may be a problem with being _too_ wordy in explaining it.
Yes, and I may have gone there ;)
Here is the current entire thing. The changes are a few more words in the Munge From and Wrap Message descriptions; adding the "If first_strip_reply_to = No" sentence to the "first_strip_reply_to = Yes" and changing the second sentence in the anonymous list paragraph.
from_is_list (general): Replace the From: header address with the list's posting address to mitigate issues stemming from the original From: domain's DMARC or similar policies.
Several protocols now in wide use attempt to ensure that use of the domain in the author's address (ie, in the From: header field) is authorized by that domain. These protocols may be incompatible with common list features such as footers, causing participating email services to bounce list traffic merely because of the address in the From: field. This has resulted in members being unsubscribed despite being perfectly able to receive mail.
The following actions are applied to all list messages when selected here. To apply these actions only to messages where the domain in the From: header is determined to use such a protocol, see the dmarc_moderation_action settings under Privacy options... -> Sender filters.
Settings:
No Do nothing special. This is appropriate for anonymous lists. It is appropriate for dedicated announcement lists, unless the From: address of authorized posters might be in a domain with a DMARC or similar policy. It is also appropriate if you choose to use dmarc_moderation_action other than Accept for this list. Munge From This action replaces the poster's address in the From: header with the list's posting address and adds the poster's address to the addresses in the original Reply-To: header. Wrap Message Just wrap the message in an outer message with the From: header containing the list's posting address and with the original From: address added to the addresses in the original Reply-To: header and with Content-Type: message/rfc822. This is effectively a one message MIME format digest.
The transformations for anonymous_list are applied before any of these actions. It is not useful to apply actions other than No to an anonymous list, and if you do so, the result may be surprising.
The Reply-To: header munging actions below interact with these actions as follows:
first_strip_reply_to = Yes will remove all the incoming Reply-To: addresses but will still add the poster's address to Reply-To: for all three settings of reply_goes_to_list which respectively will result in just the poster's address, the poster's address and the list posting address or the poster's address and the explicit reply_to_address in the outgoing Reply-To: header. If first_strip_reply_to = No the poster's address in the original From: header, if not already included in the Reply-To:, will be added to any existing Reply-To: address(es).
These actions, whether selected here or via dmarc_moderation_action, do not apply to messages in digests or archives or sent to usenet via the Mail<->News gateways.
If dmarc_moderation_action applies to this message with an action other than Accept, that action rather than this is applied
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Andrew Partan writes:
Until people figure out real ways of making DMARC work with forwrders & mailing lists (see ietf-822@ietf.org for one place discussions are going on), I think it useful to have more work-around hacks out there so that people can experiment with them to see which ones more-or-less work in different situations.
That's what they said about Reply-To munging, too.
If people want to implement them themselves and try them out, heck, I've been wrong before and I'll be wrong again. But I don't think *Mailman* should proactively implement RFC violations unless there's a clear and present danger, and then the violations should try to be minimal.[1]
DMARC, since it causes denial of service to third parties, is such a clear and present danger. Here I interpret "minimal" to mean "try to avoid to adding to the set of RFC violations out there." I know it's tempting to imply that yahoo.com is an invalid domain, but it's not at all necessary given that substituting the list-post address is what Yahoo itself suggests. The original user is easily replied to via the Reply-To hack.
Footnotes: [1] Retroactive implementation, such as "Reply-To munging", may be appropriate in response to customer demand.
On Fri, 2014-05-02 at 22:25 -0700, Mark Sapiro wrote:
On 05/01/2014 11:29 PM, Lindsay Haisley wrote:
There may be a problem with being _too_ wordy in explaining it.
Yes, and I may have gone there ;)
Well I think you've pretty well covered the bases. It may take a bit of study on the part of list admins to understand it, but this is much improved over what it was.
This is the kind of thing that people with experience in, or knowledge of structured programming can explain rather succinctly in logical pseudocode, but you're working with the more traditional prose style in which the rest of Mailman's internal docs are written, and a certain amount of redundant verbiage is probably inevitable.
-- Lindsay Haisley | "Everything works if you let it" FMP Computer Services | 512-259-1190 | --- The Roadie http://www.fmp.com |
participants (7)
-
Andrew Partan -
Larry Kuenning -
Larry Stone -
Lindsay Haisley -
Mark Sapiro -
Stephen J. Turnbull -
Stephen J. Turnbull