Avoiding mangling in Mailman 3?
All,
One of my main gripes is the From: mangling that we had to use to allow AOL and Yahoo subscribers to send messages without messing everyone else up. I have now been informed that Mailman 3 does not solve this problem, but I’d like to move to Mailman 3 anyway. So what to do?
How about this:
- Replace the From: address with a no-reply address on the list server. Don’t add the sender’s address in quotes.
- Keep the ‘Reply-To:’ address as the sender’s address (that’s what I have it set to now - I don’t want people to reply to the lists).
- Put HTML mailto: links for ‘Reply to Sender’ and Reply to List’ at the bottom of the message.
My other solution:
Require subscribers from AOL/Yahoo and whichever other service with the same misguided policy to get another email address for the lists.
Is there anyone on this who will be willing to help installing Mailman 3 for me on a Linux system. I have tried and I have had two experts try as well, but we have all run into difficulty. I’ll pay, of course.
Yours,
Allan Hansen
Allan Hansen writes:
I believe this requires a change in the Mailman code.
I guess you want the author's display name, if available, there? What if there is no display name, or it doesn't identify the author? I guess the best play would be the mailbox of the author.
How about "mailbox AT domain.com"? I guess some nontechnical users might just copy that to an address field with less than amusing results, but it might be useful in manual lookups in address books, since most clients do not display Reply-To.
I believe this doesn't need a change to Mailman.
A mailto link for the list is configurable. However, the link for author would require changes to Mailman code I'm pretty sure. Also, as explained below, it's probably very unreliable and unattractive to try to use links to simulate a mail client's reply function.
First of all, users expect a reply function to copy the text of the original. mailto URLs don't provide a facility for that. We would have to add code to copy the text to the URL. I'm not sure how typical clients would react to that, and if the original is plain text, the message's whole text would be visibly duplicated in the footer of the message distributed to subscribers, which would likely be displayed as is by most clients. This would be pretty distressing to most subscribers, I think. Also, I expect most clients use the DOM they have constructed to display the original mail to populate replies they construct themselves, but Mailman can't know about that. Users may not be pleased with replies constructed from a mailto URL; in particular, it would not be displayed or transmitted as copied, but rather as original text.
Second, users expect replies to preserve threading. This would mean adding References or at least In-Reply-To header fields to the mailto URLs. This would be straightforward to implement, but would result in large, unreadable plaintext footers, if some users are sending plaintext mail. (Though it wouldn't be as bad as if you tried to include the original text in the reply's composition window, I imagine you'd get complaints.) Again, I'm not sure how typical clients would deal with it, whether they would follow the RFCs or screw up.
Third, if you mean "links as HTML" rather than "insert URLs somehow", this is rather problematic. HTML mail is a minefield. There are standards, but in practice they're all violated by one client or another. Manipulating HTML before forwarding to the subscribers is very likely to have bad side effects for some mail composed by some clients.
You're also not guaranteed that all the subscribers even use HTML mail, in which case even if added as HTML, in those messages you'd have all the ugliness described above.
Steve
Stephen et al.,
You’re right that using links instead of a Reply function is unattractive and not how email is supposed to work. On the other hand, the same surely goes for the To: mangling:
I have set the ReplyTo: as the author, it’s not the immediate replying as such that is an issue, and the mangled string is factually correct. The issue comes when Apple Mail does auto-completion and hides the email address.
A mangled From: address like this:
"Allan Hansen (hansen@rc.org <mailto:hansen@rc.org>) via list" <rc@mail.rc.org <mailto:rc@mail.rc.org>>
will show up as
‘Allan Hansen (hansen@rc.org <mailto:hansen@rc.org>) via list’
in Apple Mail with the address hidden by the mail client. Anyone sending to that string will assume that it goes to me. It does not. It goes to the list.
So putting “Allan Hansen (hansen AT rc.org <http://rc.org/>)" in the description will not help this issue. Using it with auto-completion will still send it to the hidden list address.
You’re right that if the author is sending HTML mail, adding a hyperlink to it is not likely to be successful at all. So maybe that’s not a good solution. On the other hand, if this is the case, it appears that the automatically inserted message footer added by Mailman is working fine (see next). Would anything prevent adding to this section? Can it be a REPLY button?
------------------------------------------------------
Allan's mailing list
allan@mail.rc.org <mailto:allan@mail.rc.org>
<http://mail.rc.org/mailman/listinfo/allan <http://mail.rc.org/mailman/listinfo/allan>>I did the code change for the mangling long ago, on advice from this list. It worked for a while, until the auto-completion issue and hiding of the actual email addresses messed it up. I’d rather not have the same problems in Mailman 3, so I’m looking for something, anything - even if it’s not nice, that does not cause my subscribers to be confused or to send private messages to 1000 people without knowing it.
Yours,
AllanAllan Hansen writes:
That's not our mangling, it's your client's. Ours would look like this:
"Allan Hansen (hansen@rc.org) via list" <rc@mail.rc.org>Most likely, the client actually uses the Mailman version in composing replies but displays the above to you (or presents it to the system highlight-and-copy function).
I don't contest your statement of fact, but if Reply-To indeed contains author, it should go to author. (I believe we *always* add author to Reply-To if Munge From is in effect.) If it doesn't go to Reply-To, the client is at fault, because if Reply-To is set, it is the *author*'s preferred address for receiving replies, and From *should* be ignored in collecting the addressees to use in replies. (It's possible that it *also* goes to the list, if the list is in Reply-To or the user requests "reply to all".)
Bottom line: I'll do what I can to help you, but I'm not sure whatever it is we come up with is suitable for adding to the main Mailman distribution.
No.
Can it be a REPLY button?
No. I'm pretty sure the footer is plain text (see below) and plain text can't specify buttons. The *client* can add them as easily as it can turn addresses in text into clickable links.
The problem is that the From (if no Reply-To) and Reply-To (preferred if present) header fields are the only reliable ways to inform the client that a given address is the author's preferred reply address. In other words, if it ain't working, it's broke beyond what we can do to fix. We can maybe help or workaround, but no promises, because anything we do will require cooperation from your subscribers to be effective. We can help make the RightThang[tm] easier to discover, but the proverb about horses and water applies.
I don't know what your client is so I can't be sure, but the line of hyphens suggests that the footer is added as a text/plain part, not text/html. In any case, Mailman presents the addresses as text, not as links, so it must be something the client is doing.
It turns out to be easy for software to recognize URLs in text, so most clients do so and turn them into links automatically, even though Mailman doesn't. I'm pretty sure that this works fine in almost all clients. When it doesn't, it's normally an easy copy-paste. Adding a mailto URL to list in the footer can be done, it's just a matter of editing a template. (I don't think that editing can be done in Postorius (the admin on the web module) yet, but I believe it's on the todo list.)
Adding a mailto URL to author probably requires a feature we don't have yet (I don't think the author's address is made available to the footer generation function.) I will look into that. Again, even with this feature you would have to customize in the footer template. It will not be a default behavior of Mailman.
Without seeing a full header of a message as distributed by your list, I can't be sure. But if the settings are as you say and the From and Reply-To headers are set as I expect they are, there is no sure way to prevent misaddressed mail except to change clients, because the clients that reply to From in the presence of a Reply-To field are just plain broken. The list cannot work around them without risking breaking other clients, where those other clients *are* conforming to Internet standards.
Alternatively, you could ban posts by @yahoo.com and @aol.com addresses, in which case most of the need to munge from goes away.
But neither of those is likely to make your subscribers happy. People hate changing their mail clients and mail addresses more than anything else in computing, it seems. I can't really blame them about hating to change address, for sure, and I don't know of any mail clients that make it simple to change to a different one. There are always some data that you depend on that have to be extracted from the previous client per https://xkcd.com/538/ ;-).
BTW, I have not used EMWD myself, but Brian is a good citizen on the list, and he does due diligence on this list in terms of asking us to help with solving problems for his clients. EMWD is top of my list if I ever decide to delegate my list or servers to a commercial service.
Steve
On 12/8/19 8:17 PM, Stephen J. Turnbull wrote:
Actually, that's still not quite correct. Ours looks like
"xxx via list" <rc@mail.rc.org>
where xxx is if original From contains a display name then that display name else if original From is a member and the member has a real name then that real name else the From email address.
I.e., it could be
"Allan Hansen via list" <rc@mail.rc.org>
or
"hansen@rc.org via list" <rc@mail.rc.org>
but wouldn't be
"Allan Hansen (hansen@rc.org) via list" <rc@mail.rc.org>
I suspect this latter is due to what Allan alluded to when he said "I did the code change for the mangling long ago ...".
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro writes:
Actually, that's still not quite correct. Ours looks like
"xxx via list" <rc@mail.rc.org>
Thank you for the correction.
I wonder whether users would behave differently if we used "list on behalf of xxx" instead of "xxx via list"?
I suspect this latter is due to what Allan alluded to when he said "I did the code change for the mangling long ago ...".
Ah, I see.
Hi Stephen,
Thank you a bunch for looking into this.
I was trying to say that ReplyTo: works fine, for just the reasons you mention. No problem there. At first. ;-) But Apple Mail puts the mangled address To: into the ‘Previous Recipients’ list to help with auto-completion later.
Here are the steps. I’m avoiding real addresses, as my mail client further mangled them with the auto-inserted ‘mailto:’ command confusing my message.
Here goes:
a. Subscriber receives message from the list. The From: is a mangled From: as recommended, and the ReplyTo: is the author’s emal address: From: Author Name (author.address) via list <list.address> ReplyTo: author.address
b. Subscriber replies to author. Sees correct To: address (the author.address) from the ReplyTo: header. So far all is apparently OK. However, to be ‘helpful’ with auto-completion later, Apple puts the mangled string “Author Name (author.address) via list <list.address>” into the mail client’s ‘Previous Recipients’ list!! To: author.address
c. Subscriber much later tries to send a private message to the author and starts typing "Autho...". Apple at this point retrieves the mangled string from the ‘Previous Recipents’ list, but in their infinite wisdom, they hide the actual address, which is the list address. The subscriber does not suspect that things have gone awry because it looks fine. Well, not completely fine, but enough so. So he/she hits ’Send’ while seeing this and only this in their To: field: To: Author Name (author.address) via list
d. People on the list receive a private message that was intended for the original author. Result: red faces all around and possibly private data exposed to the entire list. I just now happened to receive such a message from one of my lists! No real disaster this time, luckily, but confusing for the lists members.
I do tell people to clean up their ‘Previous Recipients’ list, they eventually forget and this happens again.
If this can’t be solved somehow, I will have to unsub all my AOL and YAHOO subscribers (a lot), as it’s too dangerous to have the mangling causing these privacy mishaps. They don’t really have to change their main email, just get another one that they use only for the lists.
By the way, I have asked Brian to help with installing Mailman 3 and look forward to working with him and with the new system.
Yours,
Allan
This is exactly the problem I mentioned a few weeks earlier that did not elicit much of a response. I asked for some way to change the "via" string and there wasn't one. I have to remind people periodically to remove any "via" entries from their address books.
AOL/Yahoo/Verizon cause other problems too, due to server reputation. I signed up for a hosted Mailman 3 service, added a list with about 60 A/Y/V addresses, and it caused terrible server reputation problems for the provider. Mail delivery to A/Y/V was simply dropped or held up for days, and was always classified as spam. It appears to be ameliorated now, but it was a horror show. No amount of cajoling can get the users off those providers - they have enough trouble just operating their mail clients. And I cannot simply drop them: they are members of an organization the lists serve.
Dan
On 12/9/19 9:06 AM, Allan Hansen wrote:
Allan Hansen writes:
But Apple Mail puts the mangled address To: into the ‘Previous Recipients’ list to help with auto-completion later.
I assume by "To" you mean "From". We don't munge "To" in this situation (there is a personalized list configuration where To is changed from the list to the subscriber, but that shouldn't cause client problems or DMARC issues).
I don't see how we can do anything reliable about that. From is a *required* field in RFC 5322 message syntax, and it *must* contain a mailbox (perhaps along with a display name). Some possibilities follow.
We could stick a .invalid address in there, which would immediately bounce back to the sender. But that screws those users because the client hides the address and we don't control the delivery service notice in that case, it's the user's outgoing mail gateway that will reject it. Or possibly silently drop it. So there's no guarantee they'll get a message that makes any sense to them. And if they figure out the list is related, you'll take some heat.
We could put an "oopsie, did you mean to send to us" address at the Mailman host in there that replies with explanation from Mailman, but when you don't have the list in Reply-To, people who *intend* a reply to list will have to copy/paste by hand (as mentioned earlier a link in the footer will not have the features of a client-composed reply). That might be OK for you, since you seem to really discourage replies to list.
Another try would be a Rule that checks for the "via list-at-this- server" formulation and automatically bounces the mail back (regardless of any "don't at me" settings), with an explanation of why the mail bounced and a suggestion to clean up Previous Recipients. You could simulate this with the existing spam hold feature, but I'm not sure that can be set to reject on a per recipe basis, and I don't think it would allow for the explanation to differ across rejections.
Of course that will fail if the user changes the display name. What is your experience? Do these users just accept the display name with "via list" attached, or do they tend to fix it while failing to notice the unintended address? It sounds like this might be good enough, possibly combined with a second Rule which looks for the list's display name (and any variants popular with posters) and holds posts that don't contain it for moderation.
(Note to me: Possibly the mail's entire content would need to be scrubbed when bouncing but available at the archive as a .bin thingie to avoid certain kinds of bounce spam. And if so, it would need to be removed in say 24 hours since it's almost certainly private. Also maybe a check for In-Reply-To/References could help identify likely private mail?)
I do tell people to clean up their ‘Previous Recipients’ list, they eventually forget and this happens again.
You're a hero! But this sucks for you. The point of an advanced list manager is that you shouldn't have to do this kind of mechanical work.
Steve
On Dec 10, 2019, at 10:27 , Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
[ABH] Yes, sorry. It takes the “From:” address and saves that, instead of the “ReplyTo:” address that is the new “To:” address.
[ABH] The “From:” should contain the author address, but if we want to keep our Yahoo/AOL subscribers…
[ABH] That’s not a bad idea, Stephen. I could try that. And yes, we are very protective of our lists, so “Reply-To:” is the author address. When I get Mailman 3 set up, I’ll put in an ‘oopsie’ address with an auto-responder. I’ll assume that Mailman 3 will be able to detect auto-responder infinite loops. :-)
[ABH] The disasters all have had the full mangled display name, so no editing took place in those cases. I think the first suggestion above is better.
[ABH] I’m trying to get out of it, as you can see. :-)
I very much appreciate your suggestions and help, and will let you and the list know how the autoresponder works out. I’m a bit red in the face that I did not think of that, but what are friends for!
Yours
AllanAllan Hansen writes:
[ABH] The “From:” should contain the author address, but if we want to keep our Yahoo/AOL subscribers…
Exactly. This is what we economists call the Theorem of the Second Best: When it's already broken, sometimes the best you can do is to break it harder.
Without change to the Mailman code (Mailman 2 or Mailman 3), the "oopsie" address will have to be handled by a separate autoresponder. Mailman only knows about the standard addresses, and it doesn't allow configuration of autoresponder features beyond "on" and "off".
I’m a bit red in the face that I did not think of that, but what are friends for!
"Mail is hard, and then you retire." From "The Sysadmin's Lament". :-)
Steve
Allan Hansen writes:
I believe this requires a change in the Mailman code.
I guess you want the author's display name, if available, there? What if there is no display name, or it doesn't identify the author? I guess the best play would be the mailbox of the author.
How about "mailbox AT domain.com"? I guess some nontechnical users might just copy that to an address field with less than amusing results, but it might be useful in manual lookups in address books, since most clients do not display Reply-To.
I believe this doesn't need a change to Mailman.
A mailto link for the list is configurable. However, the link for author would require changes to Mailman code I'm pretty sure. Also, as explained below, it's probably very unreliable and unattractive to try to use links to simulate a mail client's reply function.
First of all, users expect a reply function to copy the text of the original. mailto URLs don't provide a facility for that. We would have to add code to copy the text to the URL. I'm not sure how typical clients would react to that, and if the original is plain text, the message's whole text would be visibly duplicated in the footer of the message distributed to subscribers, which would likely be displayed as is by most clients. This would be pretty distressing to most subscribers, I think. Also, I expect most clients use the DOM they have constructed to display the original mail to populate replies they construct themselves, but Mailman can't know about that. Users may not be pleased with replies constructed from a mailto URL; in particular, it would not be displayed or transmitted as copied, but rather as original text.
Second, users expect replies to preserve threading. This would mean adding References or at least In-Reply-To header fields to the mailto URLs. This would be straightforward to implement, but would result in large, unreadable plaintext footers, if some users are sending plaintext mail. (Though it wouldn't be as bad as if you tried to include the original text in the reply's composition window, I imagine you'd get complaints.) Again, I'm not sure how typical clients would deal with it, whether they would follow the RFCs or screw up.
Third, if you mean "links as HTML" rather than "insert URLs somehow", this is rather problematic. HTML mail is a minefield. There are standards, but in practice they're all violated by one client or another. Manipulating HTML before forwarding to the subscribers is very likely to have bad side effects for some mail composed by some clients.
You're also not guaranteed that all the subscribers even use HTML mail, in which case even if added as HTML, in those messages you'd have all the ugliness described above.
Steve
Stephen et al.,
You’re right that using links instead of a Reply function is unattractive and not how email is supposed to work. On the other hand, the same surely goes for the To: mangling:
I have set the ReplyTo: as the author, it’s not the immediate replying as such that is an issue, and the mangled string is factually correct. The issue comes when Apple Mail does auto-completion and hides the email address.
A mangled From: address like this:
"Allan Hansen (hansen@rc.org <mailto:hansen@rc.org>) via list" <rc@mail.rc.org <mailto:rc@mail.rc.org>>
will show up as
‘Allan Hansen (hansen@rc.org <mailto:hansen@rc.org>) via list’
in Apple Mail with the address hidden by the mail client. Anyone sending to that string will assume that it goes to me. It does not. It goes to the list.
So putting “Allan Hansen (hansen AT rc.org <http://rc.org/>)" in the description will not help this issue. Using it with auto-completion will still send it to the hidden list address.
You’re right that if the author is sending HTML mail, adding a hyperlink to it is not likely to be successful at all. So maybe that’s not a good solution. On the other hand, if this is the case, it appears that the automatically inserted message footer added by Mailman is working fine (see next). Would anything prevent adding to this section? Can it be a REPLY button?
------------------------------------------------------
Allan's mailing list
allan@mail.rc.org <mailto:allan@mail.rc.org>
<http://mail.rc.org/mailman/listinfo/allan <http://mail.rc.org/mailman/listinfo/allan>>I did the code change for the mangling long ago, on advice from this list. It worked for a while, until the auto-completion issue and hiding of the actual email addresses messed it up. I’d rather not have the same problems in Mailman 3, so I’m looking for something, anything - even if it’s not nice, that does not cause my subscribers to be confused or to send private messages to 1000 people without knowing it.
Yours,
AllanAllan Hansen writes:
That's not our mangling, it's your client's. Ours would look like this:
"Allan Hansen (hansen@rc.org) via list" <rc@mail.rc.org>Most likely, the client actually uses the Mailman version in composing replies but displays the above to you (or presents it to the system highlight-and-copy function).
I don't contest your statement of fact, but if Reply-To indeed contains author, it should go to author. (I believe we *always* add author to Reply-To if Munge From is in effect.) If it doesn't go to Reply-To, the client is at fault, because if Reply-To is set, it is the *author*'s preferred address for receiving replies, and From *should* be ignored in collecting the addressees to use in replies. (It's possible that it *also* goes to the list, if the list is in Reply-To or the user requests "reply to all".)
Bottom line: I'll do what I can to help you, but I'm not sure whatever it is we come up with is suitable for adding to the main Mailman distribution.
No.
Can it be a REPLY button?
No. I'm pretty sure the footer is plain text (see below) and plain text can't specify buttons. The *client* can add them as easily as it can turn addresses in text into clickable links.
The problem is that the From (if no Reply-To) and Reply-To (preferred if present) header fields are the only reliable ways to inform the client that a given address is the author's preferred reply address. In other words, if it ain't working, it's broke beyond what we can do to fix. We can maybe help or workaround, but no promises, because anything we do will require cooperation from your subscribers to be effective. We can help make the RightThang[tm] easier to discover, but the proverb about horses and water applies.
I don't know what your client is so I can't be sure, but the line of hyphens suggests that the footer is added as a text/plain part, not text/html. In any case, Mailman presents the addresses as text, not as links, so it must be something the client is doing.
It turns out to be easy for software to recognize URLs in text, so most clients do so and turn them into links automatically, even though Mailman doesn't. I'm pretty sure that this works fine in almost all clients. When it doesn't, it's normally an easy copy-paste. Adding a mailto URL to list in the footer can be done, it's just a matter of editing a template. (I don't think that editing can be done in Postorius (the admin on the web module) yet, but I believe it's on the todo list.)
Adding a mailto URL to author probably requires a feature we don't have yet (I don't think the author's address is made available to the footer generation function.) I will look into that. Again, even with this feature you would have to customize in the footer template. It will not be a default behavior of Mailman.
Without seeing a full header of a message as distributed by your list, I can't be sure. But if the settings are as you say and the From and Reply-To headers are set as I expect they are, there is no sure way to prevent misaddressed mail except to change clients, because the clients that reply to From in the presence of a Reply-To field are just plain broken. The list cannot work around them without risking breaking other clients, where those other clients *are* conforming to Internet standards.
Alternatively, you could ban posts by @yahoo.com and @aol.com addresses, in which case most of the need to munge from goes away.
But neither of those is likely to make your subscribers happy. People hate changing their mail clients and mail addresses more than anything else in computing, it seems. I can't really blame them about hating to change address, for sure, and I don't know of any mail clients that make it simple to change to a different one. There are always some data that you depend on that have to be extracted from the previous client per https://xkcd.com/538/ ;-).
BTW, I have not used EMWD myself, but Brian is a good citizen on the list, and he does due diligence on this list in terms of asking us to help with solving problems for his clients. EMWD is top of my list if I ever decide to delegate my list or servers to a commercial service.
Steve
On 12/8/19 8:17 PM, Stephen J. Turnbull wrote:
Actually, that's still not quite correct. Ours looks like
"xxx via list" <rc@mail.rc.org>
where xxx is if original From contains a display name then that display name else if original From is a member and the member has a real name then that real name else the From email address.
I.e., it could be
"Allan Hansen via list" <rc@mail.rc.org>
or
"hansen@rc.org via list" <rc@mail.rc.org>
but wouldn't be
"Allan Hansen (hansen@rc.org) via list" <rc@mail.rc.org>
I suspect this latter is due to what Allan alluded to when he said "I did the code change for the mangling long ago ...".
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro writes:
Actually, that's still not quite correct. Ours looks like
"xxx via list" <rc@mail.rc.org>
Thank you for the correction.
I wonder whether users would behave differently if we used "list on behalf of xxx" instead of "xxx via list"?
I suspect this latter is due to what Allan alluded to when he said "I did the code change for the mangling long ago ...".
Ah, I see.
Hi Stephen,
Thank you a bunch for looking into this.
I was trying to say that ReplyTo: works fine, for just the reasons you mention. No problem there. At first. ;-) But Apple Mail puts the mangled address To: into the ‘Previous Recipients’ list to help with auto-completion later.
Here are the steps. I’m avoiding real addresses, as my mail client further mangled them with the auto-inserted ‘mailto:’ command confusing my message.
Here goes:
a. Subscriber receives message from the list. The From: is a mangled From: as recommended, and the ReplyTo: is the author’s emal address: From: Author Name (author.address) via list <list.address> ReplyTo: author.address
b. Subscriber replies to author. Sees correct To: address (the author.address) from the ReplyTo: header. So far all is apparently OK. However, to be ‘helpful’ with auto-completion later, Apple puts the mangled string “Author Name (author.address) via list <list.address>” into the mail client’s ‘Previous Recipients’ list!! To: author.address
c. Subscriber much later tries to send a private message to the author and starts typing "Autho...". Apple at this point retrieves the mangled string from the ‘Previous Recipents’ list, but in their infinite wisdom, they hide the actual address, which is the list address. The subscriber does not suspect that things have gone awry because it looks fine. Well, not completely fine, but enough so. So he/she hits ’Send’ while seeing this and only this in their To: field: To: Author Name (author.address) via list
d. People on the list receive a private message that was intended for the original author. Result: red faces all around and possibly private data exposed to the entire list. I just now happened to receive such a message from one of my lists! No real disaster this time, luckily, but confusing for the lists members.
I do tell people to clean up their ‘Previous Recipients’ list, they eventually forget and this happens again.
If this can’t be solved somehow, I will have to unsub all my AOL and YAHOO subscribers (a lot), as it’s too dangerous to have the mangling causing these privacy mishaps. They don’t really have to change their main email, just get another one that they use only for the lists.
By the way, I have asked Brian to help with installing Mailman 3 and look forward to working with him and with the new system.
Yours,
AllanThis is exactly the problem I mentioned a few weeks earlier that did not elicit much of a response. I asked for some way to change the "via" string and there wasn't one. I have to remind people periodically to remove any "via" entries from their address books.
AOL/Yahoo/Verizon cause other problems too, due to server reputation. I signed up for a hosted Mailman 3 service, added a list with about 60 A/Y/V addresses, and it caused terrible server reputation problems for the provider. Mail delivery to A/Y/V was simply dropped or held up for days, and was always classified as spam. It appears to be ameliorated now, but it was a horror show. No amount of cajoling can get the users off those providers - they have enough trouble just operating their mail clients. And I cannot simply drop them: they are members of an organization the lists serve.
Dan
On 12/9/19 9:06 AM, Allan Hansen wrote:
Allan Hansen writes:
But Apple Mail puts the mangled address To: into the ‘Previous Recipients’ list to help with auto-completion later.
I assume by "To" you mean "From". We don't munge "To" in this situation (there is a personalized list configuration where To is changed from the list to the subscriber, but that shouldn't cause client problems or DMARC issues).
I don't see how we can do anything reliable about that. From is a *required* field in RFC 5322 message syntax, and it *must* contain a mailbox (perhaps along with a display name). Some possibilities follow.
We could stick a .invalid address in there, which would immediately bounce back to the sender. But that screws those users because the client hides the address and we don't control the delivery service notice in that case, it's the user's outgoing mail gateway that will reject it. Or possibly silently drop it. So there's no guarantee they'll get a message that makes any sense to them. And if they figure out the list is related, you'll take some heat.
We could put an "oopsie, did you mean to send to us" address at the Mailman host in there that replies with explanation from Mailman, but when you don't have the list in Reply-To, people who *intend* a reply to list will have to copy/paste by hand (as mentioned earlier a link in the footer will not have the features of a client-composed reply). That might be OK for you, since you seem to really discourage replies to list.
Another try would be a Rule that checks for the "via list-at-this- server" formulation and automatically bounces the mail back (regardless of any "don't at me" settings), with an explanation of why the mail bounced and a suggestion to clean up Previous Recipients. You could simulate this with the existing spam hold feature, but I'm not sure that can be set to reject on a per recipe basis, and I don't think it would allow for the explanation to differ across rejections.
Of course that will fail if the user changes the display name. What is your experience? Do these users just accept the display name with "via list" attached, or do they tend to fix it while failing to notice the unintended address? It sounds like this might be good enough, possibly combined with a second Rule which looks for the list's display name (and any variants popular with posters) and holds posts that don't contain it for moderation.
(Note to me: Possibly the mail's entire content would need to be scrubbed when bouncing but available at the archive as a .bin thingie to avoid certain kinds of bounce spam. And if so, it would need to be removed in say 24 hours since it's almost certainly private. Also maybe a check for In-Reply-To/References could help identify likely private mail?)
I do tell people to clean up their ‘Previous Recipients’ list, they eventually forget and this happens again.
You're a hero! But this sucks for you. The point of an advanced list manager is that you shouldn't have to do this kind of mechanical work.
Steve
On Dec 10, 2019, at 10:27 , Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
[ABH] Yes, sorry. It takes the “From:” address and saves that, instead of the “ReplyTo:” address that is the new “To:” address.
[ABH] The “From:” should contain the author address, but if we want to keep our Yahoo/AOL subscribers…
[ABH] That’s not a bad idea, Stephen. I could try that. And yes, we are very protective of our lists, so “Reply-To:” is the author address. When I get Mailman 3 set up, I’ll put in an ‘oopsie’ address with an auto-responder. I’ll assume that Mailman 3 will be able to detect auto-responder infinite loops. :-)
[ABH] The disasters all have had the full mangled display name, so no editing took place in those cases. I think the first suggestion above is better.
[ABH] I’m trying to get out of it, as you can see. :-)
I very much appreciate your suggestions and help, and will let you and the list know how the autoresponder works out. I’m a bit red in the face that I did not think of that, but what are friends for!
Yours
AllanAllan Hansen writes:
[ABH] The “From:” should contain the author address, but if we want to keep our Yahoo/AOL subscribers…
Exactly. This is what we economists call the Theorem of the Second Best: When it's already broken, sometimes the best you can do is to break it harder.
Without change to the Mailman code (Mailman 2 or Mailman 3), the "oopsie" address will have to be handled by a separate autoresponder. Mailman only knows about the standard addresses, and it doesn't allow configuration of autoresponder features beyond "on" and "off".
I’m a bit red in the face that I did not think of that, but what are friends for!
"Mail is hard, and then you retire." From "The Sysadmin's Lament". :-)
Steve
participants (5)
-
Allan Hansen -
Brian Carpenter -
Dan Halbert -
Mark Sapiro -
Stephen J. Turnbull