[Mailman-Developers] Reply-To munging considered *carefully*

[Michael B. Trausch]

On Wed, 2009-10-14 at 17:30 +1100, Ben Finney wrote:
> On 14-Oct-2009, Michael B. Trausch wrote:
> > On Wed, 2009-10-14 at 12:55 +0900, Stephen J. Turnbull wrote:
> > > Reply-To still is truly broken.  The author wants personal replies
> > > to go to her, but now they go to the list.  The recipient must
> > > *specifically avoid reply-to-author* in order to reply to the
> > > author. This is so Orwellian.
> > 
> > Permit me to rephrase so that you understand what I said:
> > 
> > There are two major uses of email [for actual discussion]:  email
> > from one person or to one person or a group of people, and email
> > from one person to a mailing list (presumably with people on it).
> 
> I can't parse the above into two; I assume there's a mistake, and that
> you meant “email from one person to one or more people, and email from
> one person to a mailing list”. If you meant something else, I'm not
> seeing it.

No, that's exactly what I meant; I'm not sure where the mis-read is
happening, because what you said and what I said are exactly the same.

> 
> > However, the primary use-case for a mailing list is _group_
> > _discussion_.
> 
> Yes. This majority use, though, should not make it more difficult to
> use “reply to author”.

I think that there are some differences in base assumptions here.  The
initial comment that started this whole regrettable mess was that my
software needs to do what I mean.  I'm utterly _sick_ of the
inconsistency and it's becoming a real drag on getting things done,
because of late I have to wonder if I really sent the blasted messages
correctly.

We have:

 * MLs that allow a MUA to click reply and _just_ reply to the ML, and
 * MLs that require "reply all" instead, and instead of sending _one_
   message in threading context, send N number of messages where N is
   however many people are listed in various headers.

Now, _this_ is where the situation is _bad_.  It's _awful_.  If every
single ML worked in precisely the same way, there would be _zero_
issues.  Yes, some failure modes would exist that wouldn't in the other
one and whatever, but at the very least, the modes of operation would be
_consistent_.  In such a world, the idea of "do what I mean" is
something that we can then teach our systems to do for us.  Presently,
it is not.

Now, we have in the first category, the "click reply to reply to the
message" lists.  These lists save bandwidth and make things easy for the
_vast_ majority of use cases by not requiring the user to track nearly
anything.  They're automatic, and they Just Work.  There is, of course,
the potential downside of someone sending information to the list that
they did not intend to do so.  That is certainly a valid risk.  Is it
annoying?  Yes.  Could it potentially be disastrous?  Yes, though most
often not because with these types of lists most risk is easily
mitigated because it's caught early enough to have something done about
it through one of many channels.

On the other hand, we have lists that don't do that.  And for those, we
must have MUAs send multiple messages; at minimum, two: one to the
author of the message being replied to, and one as a Cc: to the mailing
list.  This introduces failure modes that are _consistently_ annoying
and not easily automatically dealt with, ranging from issues like broken
mail filters (because some lists don't have a [list] appendage in the
subject and thus one must filter on a List-* header, which is not
present when you are in To: or Cc: _and_ the message is sent to the ML),
to not posting to the list at all.  And I'm far from the only one that
has encountered these frustrations recently on various MLs.

How can these be handled?

Again, let's look at the first type of list.  There is a possible
automatic solution, and (more likely to be used on lists where it could
be a persistent issue) a less automatic means employing convention.  The
automatic solution would be MUA-side, not ML-server side, in that MUA
software would prompt the user ("How do you wish to reply? [To List] [To
Sender] [To All]").  The user could choose one to be the default if
desired, and then the burden of responsibility is (rightly) on them.
This is similar to what Alpine implements, though better.  There are a
couple of other automatic solutions, as well, which I will only mention
in brief, because I don't think anyone really cares about them:

 * A List-Hold-For MUA header with a sane maximum value (maybe 1800
   seconds).  ML could send a reply that users could act on (meant to,
   didn't mean to) and if it times out based on the header, the ML would
   go ahead and post the message.  A Web UI could also work.

 * A List-No-Archive MUA header, which clients could set by default if
   desired which would confine leakages to recipients and not archives
   or mail/nntp gateways, though the downsides of that would be obvious.

 * A List-User-Confirmed MUA header that could be set by user agents in
   combination with a prompt from above, and the ML server could send
   an autoreply to a message with L-U-C not set and ask "Did you mean
   to do that," with a timeout of 1 day or so and a default action of
   not posting the message if no action is taken before its timeout.

 * A List-Never MUA header that would cause ML software to reject
   posts, combined with an autoreply to confirm that they really wanted
   to do that, like ML software could do in the absence of a L-U-C
   header.  MUA software could set/not set this option by configuration
   items.

 * Any of the sorts of ideas above that generate an autoreply would be
   useful with state:  The ML server could add a List-AutoReply header
   that the MUA could use to present a better UI to the user instead of
   displaying the message, with List-AutoReply-Approve and
   List-AutoReply-Deny headers specifying a URL that can be handled by
   the MUA software (or passed to an external program, say a Web
   browser) when the user wants to approve or deny a post.  Autoreplies
   would also have the original message as-would-be-posted attached.

There are, of course, other variations on any of those ideas that could
be put into practical use.  The best solutions would depend on zero MUA
features, while providing more featureful MUA software with the ability
to better handle many of the situations.  Autoreplies in a knowledgeable
MUA could be hidden entirely and just presented to the user as UI
dialogs or windows or similar.

Another solution would be to have a dedicated address for certain types
of sensitive email.  Say, an email address for a team or private,
non-archived mailing list that accepts posts from non-subscribers and is
handled by people who are responsible and won't leak things.  Then, when
someone says, "Please send the details to private-email at example.org",
it's clear what is being asked and where it is to be sent.  After all,
we _can_ assume that humans are, while imperfect, capable of being
responsible for their actions and reading the messages that are sent to
them.  If we cannot make that assumption, then there really isn't much
point to a messaging system of any sort, much less a widescale one like
email, is there?

In the second type of list, sure you have private replies by default,
but you then make _everyone_ on the list have to reply-all or whatever.
Additionally, you make _everyone_ on the list try to find ways to filter
their list posts and replies into a folder, and many MUAs are not
featureful enough to try to compare incoming messages with a references
or in-reply-to header to previous messages for the mailing list, and
even so, it'd be error prone (what if the referenced messages are
deleted?) and kludgey.  Are such lists useful?  Of course, but they are
much more work for everyone involved when compared to the simplicity of
the other type of list.  Because it is certain that not 100% of the mail
will pass through the mail server, no automatic features can be used to
help the user out either on the ML server side or on the MUA client
side, because the MUA does not have enough information to properly do
what the user wants it to do at least some of the time.

> As has no doubt already been pointed out in this discussion, the
> failure modes are wildly unbalanced:
> 
> * “Whoops, I sent a message intended for the whole list instead as a
>   reply to the author. Oh well, I'll re-send the reply to the list.”
>   Annoying for the two people involved, but no permanent harm done.
> 
> * “Whoops, I sent a message intended only for the author instead as a
>   reply to the whole list.” Completely the opposite of what “reply to
>   author” is supposed to do, and unrecoverable.

What MUA are you using that has a "Reply to Author" option?  Most that I
know have "Reply" (undecorated) or "Reply to Sender" which could be read
one of two ways.  While "Reply to Author" is always 100% unambiguous, I
don't know that many MUAs advertise their reply that way.  Evolution,
for example, presents the man UI as simply "Reply" in the context of a
message; in the menu it shows "Reply to Sender".  Poll 100 end-user
types (non-technical people) and you'll likely find 3 answers to the
question "What does reply to sender mean in the context of a mailing
list post?", with no way to deterministically predict what the
percentages are on that.*

> >  * I send an email to the mailing list to start a thread of
> >  discussion. I send no reply-to header on the message, so the list
> >  processing software appends one redirecting replies to the group.
> 
> It directs the “reply to author” replies to the group, despite the
> intent of the respondent. A recipe for disaster, as has been
> demonstrated countless times over the years.

Though, as I mentioned above, there are automatic ways to handle those
situations that can fend off such things, unlike in the other situation
where such behavior is impossible because of lack of data at least some
of the time.  Some of the means can still be applied to the second type
of list, but they don't make sense there because that just adds crap to
jump through.

The ultimate point here is that it should be _in the user's control_,
not the ML admin's control.  Certainly, the ML admin might want to say
"No, I won't remove your accidentally posted sensitive information," and
the ML admin has no liability to do so.  But with any of the safeguards
above in place, a ML admin could say that with confidence that the ML
users have to deliberately act to post in the first place.  Say the
timeout method, List-Hold-For, would likely be most useful with a short
value (say, 180 seconds by default), because most users realize that
they screwed up _immediately_ after pressing the Send button.  And guess
what?  That functionality then lets them cancel their post---putting
control of a user's owned words/data in the hands of the user, and
lightening the load of the ML administrator.  Again, for lists that
replies never go through automatically, that's simply not possible.

So if we're going to talk about mitigating leakage issues here?  The
first type of list with the appropriate safeguards in place that the
user can control with simple MUA-side headers and potentially additional
UI enhancements on the MUA itself is the _clear_ winner.  Somehow,
though, I think that this conversation isn't about such things; I think
it's about less concrete things altogether.

> > This facilitates group discussion.  It's not Orwellian, it's common
> > sense
> 
> The Orwellian behaviour is that “reply to author” has been perverted
> to mean something *totally* different.

I don't get the impression that we're talking about users here, at all.
I get the impression that we're going "Look!  See?  Bad!  We know this
to be true!  What on earth are you thinking‽", instead.  I'm speaking in
practical terms, from someone who uses software (and indeed, writes
software, administers software, etc., so I'm not a non-technical end
user).  Using words like "perverted" makes it sound like, "I'm right,
you're wrong, now go away."

The standard is ambiguous.  I'm sorry if you don't like that, but that
_is_ the way it is.  On top of the ambiguity there, I think that there
is an assumption that whatever user interface that you're using that
says "Reply to Author" is some sort of common thing among all MUA UIs,
and it's not (probably _because_ the standards are ambiguous, though it
could also be because the wording chosen most popularly is slightly more
abstract anyway, even if the authors of the MUA don't see that Reply-To
can apply to a mail from a ML server just as much as it can from a human
directly).

Again, I'm _not_ saying that ML software should override the reply-to
header if it's already present in the message (though they do tend to do
that anyway, and there's nothing we're going to change here on that).
Fact is, they _do_, and that can be turned into something that can be
used to prevent the very sorts of user mistakes being talked about here.
Why it hasn't been before is beyond me.

> For “facilitate group discussion”, we have the “reply to list”
> operation — implemented in most modern mainstream MUAs, and plenty of
> pressure on the few recalcitrant vendors to get it right.
> 
> >  * During the course of the thread, it becomes clear that some sensitive
> > information must be sent (to address your use case from your Web page
> > from earlier in this thread).  That's totally not a problem:  The person
> > asking for the sensitive information simply adds a reply-to header to
> > their outbound message.
> 
> This picture is so contrived I can't think of any situation where it's
> actually played out as you portray it. Rather, the *person who wants
> to send the private information* is the first one to know that's what
> is needed — so they hit “reply to author” to send a reply to the
> author of the message they're looking at.

I forced nothing.  The picture was posited earlier in the thread.  I
suggested a means of dealing with it, that's all.  Read further into it
if you want, but I don't warrant anything you read beyond the text...

> Under your proposal, their intent is betrayed — deliberately, by the
> mailing list software. Even though the mailing list software is *also*
> making available the standard RFC 2369 information facilitating group
> discussion.

You are forgetting that RFC 2369 can _only_ apply if the mail goes
through the ML server in the _first_ place.

Forgive me, y'all, for thinking I could talk from a user's point of
view.  I suppose I should know better.

	--- Mike

* The answers being, "I don't know," "It means reply to the mailing
list," and "It means reply to the person who sent it to the mailing
list," or variants thereof.  Because people are generally special, you
may get more than 3 answers that are "creative".

-- 
Blog:  http://mike.trausch.us/blog/
Misc. Software:  http://mike.trausch.us/software/

The FSF merely is returning freedom to software, it did not invent it.
But, what of history? Someday the FSF will go, a new proprietary era
will come, and someone else will invent freedom in software for the
third time.