[Mailman-Developers] Message threading - requesting some wisdom

[Stephen J. Turnbull]

Andrew Stuart writes:

 > I assume a great deal of thought has gone into message threading
 > and how it works. Is there anyone willing to share the basics with
 > me - specifically as it relates to Mailman and related projects?

Currently Mailman and Postorius don't care at all.  HyperKitty has its
own threading mechanism, and this is a core function of an MUA or
archive (which is sort of a read-only MUA).  AFAICS, Jamie's
threading.html is all you really need to know about this.  There's
also the IMAP THREAD extension, an alternative specification of
Jamie's algorithm: https://tools.ietf.org/html/rfc5256.

 > a Mailman oriented understanding

I don't understand what you mean by this.  Mailman's current
distribution and administration functions don't care about threads at
all, and (as I alluded to above) HyperKitty is just a read-only MUA
for this purpose.

Dynamic sublists (as implemented by the Systers fork) and some
proposals for anonymous lists using threading information, but Systers
does it by creating a separate channel (which may have subthreads
defined by links) while the anonymous list use case is as yet
undesigned.

 > example, this description of Gmail threading
 > http://xkahn.zoned.net/software/evolution/threads/ seems to suggest
 > that Gmail includes arbitrary business rules for threading based on
 > message content (i.e. subject prefix).

Jamie's algorithm also uses the subject field when links are unavailable.

 > ** What data defines how emails are connected to one another?

The In-Reply-To and References header fields are references to earlier
messages in the thread.  Most MUAs also create pseudo-threads by
sorting on Subject and Date when those fields are unavailable.

 > ** How is that connecting data organised (i.e. in a tree, a flat
 >    list, by date, whatever)?

Organized where?  Conceptually it's a tree.

 > ** What happens when thread data is missing (i.e. a message in a
 >    thread is deleted)?

Implementation detail.

 > ** Are there any algorithms/mechanisms/patterns that are effective
 >    for implementing threading?

The two documents referenced above.

 > ** Any code in Mailman/Hyperkitty/elsewhere that is particularly
 >    good to study?

HyperKitty must do threading, but what algorithm it uses I don't
know.  I find that mailing lists are generally not very demanding of
the threading algorithm, most work fine.