Camera-ready option to mitigate DMARC issues
Dear all, I'd like to probe the feasibility of this option.
The idea is to add a footer only in case it is not present, similar to what is done with subject_prefix. By properly setting both of them, a sender can submit what can be called a camera-ready post. Since no change applies, no DKIM signature breaks. Hence, dmarc_moderation_action is not needed for such posts. It is not even necessary to check author's domain policy.
I'm not familiar with Mailman administration, so I ask your opinion. How long would it take to code this option? How useful would it be?
Camera-ready posts would be created by hands, by cleverly configuring some email client, or by using purposely written add-ons. They could also be done by MSAs who care about the damage they cause by publishing p=reject --the process can certainly be standardized and automated.
TIA for any reply, and thanks for a great product Ale
On 11/05/2016 04:11 AM, Alessandro Vesely wrote:
The idea is to add a footer only in case it is not present, similar to what is done with subject_prefix. By properly setting both of them, a sender can submit what can be called a camera-ready post. Since no change applies, no DKIM signature breaks. Hence, dmarc_moderation_action is not needed for such posts. It is not even necessary to check author's domain policy.
Mailman could conceivably keep track of whether it has changed any headers or anything in the body of the post, but it's more complicated than that. The first big problem is the Munge From or Wrap Message transformations are applied before any msg_header or msg_footer is added (or maybe added).
I.e. in both MM 2.1 and MM 3, the DMARC mitigations are applied in the incoming handler pipeline before the message is queued for delivery processing. Various decorations such as adding msg_header and msg_footer and modifying To: depend on "personalization" and have to be done by delivery processing on a per-recipient basis. In fact, the "camera ready" notion can't apply to any post that is going to be personalized in any way. This in itself would limit the usefulness.
It would be more feasible to do this by the poster adding a "X-Camera-Ready:" header to the post saying don't transform my message, but this is unacceptable as it would bypass content filtering, personalization and various other things.
I'm not familiar with Mailman administration, so I ask your opinion. How long would it take to code this option?
How many angels can dance on the head of a pin?
How useful would it be?
In my opinion, certainly not enough to justify the effort in trying and the inevitable bug reports that would follow from all the edge cases.
Camera-ready posts would be created by hands, by cleverly configuring some email client, or by using purposely written add-ons. They could also be done by MSAs who care about the damage they cause by publishing p=reject --the process can certainly be standardized and automated.
How does the sender's automated process even know what msg_header and msg_footer will be added by the list?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Sat 05/Nov/2016 19:51:13 +0100 Mark Sapiro wrote:
On 11/05/2016 04:11 AM, Alessandro Vesely wrote:
The idea is to add a footer only in case it is not present, similar to what is done with subject_prefix. By properly setting both of them, a sender can submit what can be called a camera-ready post. Since no change applies, no DKIM signature breaks. Hence, dmarc_moderation_action is not needed for such posts. It is not even necessary to check author's domain policy.
Mailman could conceivably keep track of whether it has changed any headers or anything in the body of the post, but it's more complicated than that. The first big problem is the Munge From or Wrap Message transformations are applied before any msg_header or msg_footer is added (or maybe added).
Is it possible to abort processing the in-memory msg and revert to the file? Doing so --after thorough checks-- would prevent breaking the signature by altering the order of recipient, switching MIME values from token to quoted-string or vice-versa, and similar changes that memory representation may unwittingly imply.
I.e. in both MM 2.1 and MM 3, the DMARC mitigations are applied in the incoming handler pipeline before the message is queued for delivery processing.
All right, so we cannot save that lookup.
Various decorations such as adding msg_header and msg_footer and modifying To: depend on "personalization" and have to be done by delivery processing on a per-recipient basis. In fact, the "camera ready" notion can't apply to any post that is going to be personalized in any way. This in itself would limit the usefulness.
Sure, personalization cannot be compatible with camera-ready.
It would be more feasible to do this by the poster adding a "X-Camera-Ready:" header to the post saying don't transform my message, but this is unacceptable as it would bypass content filtering, personalization and various other things.
X-Camera-Ready: may be useful to automate at senders'. For an author doing it by hands, having to set a header field is an added difficulty...
I'm not familiar with Mailman administration, so I ask your opinion. How long would it take to code this option?
How many angels can dance on the head of a pin?
Ah, not so many of them are still able to perform that, nowadays ;-)
Camera-ready posts would be created by hands, by cleverly configuring some email client, or by using purposely written add-ons. They could also be done by MSAs who care about the damage they cause by publishing p=reject --the process can certainly be standardized and automated.
How does the sender's automated process even know what msg_header and msg_footer will be added by the list?
MTAs can learn List-Post addresses when they receive mail. When they see one, they can, say, change the envelope recipient to an internal mailbox which processes the decoration (and maybe adds X-Camera-Ready:). The decorating module could be carved out from Mailman, and complemented so as to let it download its parameters from well known locations or some such.
Ale
Alessandro Vesely writes:
The idea is to add a footer only in case it is not present,
Aside from the technical difficulties that Mark describes, this suffers from a really big defect: for this to be actually useful, you'd need near-100% participation (Authenticated Received Chain has the same problem). Once you've got that, no new Mailman code is needed: just don't use a footer in the first place! And it requires effort on the part of providers we already know are irresponsible and inconsiderate because they provide personal mail services disrupted by "p=reject", or on the part of users at those sites, many of whom blame mailing lists, not their irresponsible and/or inexpert providers, for difficulties with posting.
If you don't get near 100%, then you need to provide the workarounds for all users and originating sites that don't participate anyway, to all of your subscribers -- and "nonparticipants" will include the posters who are most likely to blame the lists for any delivery problems, and get upset about "different treatment" (eg, From- munging).
On Sun 06/Nov/2016 09:17:53 +0100 Stephen J. Turnbull wrote:
Alessandro Vesely writes:
The idea is to add a footer only in case it is not present,
Aside from the technical difficulties that Mark describes, this suffers from a really big defect: for this to be actually useful, you'd need near-100% participation (Authenticated Received Chain has the same problem).
No, the difference is that ARC applies at the receiving side, so you need 100% compliance of list subscribers. Camera-ready applies at the sending side, so the list can still apply any anti-DMARC workarounds if it fails.
Once you've got that, no new Mailman code is needed: just don't use a footer in the first place!
And it requires effort on the part of providers we already know are irresponsible and inconsiderate because they provide personal mail services disrupted by "p=reject", or on the part of users at those sites, many of whom blame mailing lists, not their irresponsible and/or inexpert providers, for difficulties with posting.
Yes, but then it would be less disgusting to punish users of intractable providers.
If you don't get near 100%, then you need to provide the workarounds for all users and originating sites that don't participate anyway, to all of your subscribers -- and "nonparticipants" will include the posters who are most likely to blame the lists for any delivery problems, and get upset about "different treatment" (eg, From- munging).
Yes, that's already in place. It is considered a somewhat dirty solution. Camera-ready is cleaner than anything I have heard till now. Probably it is not workable, but I cannot understand why. It works well in several publishing environments, typically journals, which distribute templates to authors. Why can't it work for mailing lists?
Ale
On 11/06/2016 09:58 AM, Alessandro Vesely wrote:
Camera-ready is cleaner than anything I have heard till now. Probably it is not workable, but I cannot understand why. It works well in several publishing environments, typically journals, which distribute templates to authors. Why can't it work for mailing lists?
Mailing lists are different. They are subject to attack from spammers and others which is different from the environments you are looking at.
I don't think your suggestions are workable. If you were to submit an actual implementation, we could evaluate it's specific feasibility, but in the absence of that, I won't spend time looking for flaws in something that doesn't exist.
Note that Mailman lists can already be configured to make no DKIM signature breaking modifications to messages. Most lists are not configured that way because the modifications, which go beyond the simple addition of a footer or a subject prefix, are desired by the list.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Alessandro Vesely writes:
Camera-ready is cleaner than anything I have heard till now.
Clean, maybe, but Mark explained why the scheme is fragile even if you can get participation. The killer problem is personalization, but the other problems are also intractable. Serial numbers in the Subject is a case where the variable content is not predictable by the author or originating system.
Probably it is not workable, but I cannot understand why. It works well in several publishing environments, typically journals, which distribute templates to authors. Why can't it work for mailing lists?
It can, it just depends on high participation rates to be useful. As I pointed out, if you can get that participation, you don't need any changes to Mailman, you just don't add footers to the body or list tags to the Subject.
So why don't you try the experiment on some of your lists and see how much participation you get?
participants (3)
-
Alessandro Vesely -
Mark Sapiro -
Stephen J. Turnbull