Fine grained subscription control, scaling, noise, relevancy, subscription rules, dynamic lists
Refer to the bottom of this email for some previous quotes from @barry on this topic. I’ve also had off list discussions with Barry in which he has mentioned this same topic so it seems to have some previous thought gone into it.
I’m wanting to raise the topic of “fine grained subscription control” (for want of a better term) for discussion. Please note these thoughts refer to discussion style lists rather than notification style lists. Please prefix every sentence here with IMO - I’m not saying I’m right about anything, just putting forward food for thought. Also, I’m no Mailman expert so if my assumptions are plain wrong please let me know.
One of the core problems of mailing lists (at least as implemented in Mailman) is that there’s not much middle ground between being very involved (i.e receive all posts to list), or being almost-not-involved (i.e. receiving daily digests or no digest).
List noise and relevancy is the main problem and it’s a big impediment to lists being more widely used.
Very engaged users might be ok with constantly deleting messages that they are not interested in (i.e. irrelevant noise). Less engaged users will be annoyed with the noise and unsubscribe, or will switch to digest notification which I suggest to you is close to unsubscription anyway because that user no longer sees emails that they might have been interested in had they known there was a message on that subject. Digest users I suggest are at best observers rather than participators.
The more active the list, the greater the noise problem, the more users will drop out of the list. As a list gains more subscribers, eventually even the most engaged user will have had enough of the volume of messages and will drop back to digest or write some mailbox rule that moves the emails to a folder, effectively dis-engaging them from the list conversation flow. Thus Mailman discussion-style lists currently don’t scale.
Even for low volume lists, noise is a major inhibitor to usage in certain contexts. Consider for example the CIO of a company or the manager of a division of 30 developers. There are various mailing lists being used by the various projects that they are responsible for. The leader is not participating however because the noise from those lists would be overwhelming. They would however like to be partipcating in list discussions either that they initiated, are explicitly copied into, or relate to topics (keywords) that they are interested in. In reality, I’m not convinced Mailman would often be used in contexts like this currently because the relevancy/noise/digest/unsubscribe problem is a showstopper.
The systers have recognised this problem and their solution is Dynamic sublists as described here: http://wiki.list.org/DEV/Dynamic%20Sublists "List subscribers can decide whether to be a part of new conversations or not. If the users decide to subscribe to new conversations, then they will receive all the messages of a new conversation unless they explicitly unsubscribe from it. If they otherwise decide to unsubscribe from new conversations,then they will receive only the first message of every new conversation unless they explicitly subscribe to it.”
I don’t think dynamic sublists are an effective solution (IMO, remember?). Getting the root message in all conversations is still too noisy and its a clumsy mechanic to have to opt in to further discussion and my thinking is that opt-in-to-discuss will impose a barrier to engagement that will reduce user interaction - put another way - “opt in to every discussion? too hard”.
Possibly a solution worth considering is fine-grained subscription control, in which there is a set of SEND and DONTSEND rules at a List default level and at a user level.
SEND: Discussions where list member email address is in to/from/cc: Discussions where to/from/cc contains one or more of: sally@example.org, davina@example.org, *.example.com Discussions where subject contains keyword: hyperkitty Discussions where body contains keyword: hyperkitty
DONTSEND: Discussions where list member email address is in to/from/cc: Discussions where to/from/cc contains one or more of: sally@example.org, davina@example.org Discussions where subject contains keyword: Java Discussions where body contains keyword: ruby
The technical hurdle to making this work is that Mailman needs access to historical messages to make it work (i.e. integrating some level of aqrchive like functionality into the Mailman database). I suggest this this may not be as hard as it sounds and hey, we’ve got a database there anyway so why not use it?
One possibility is that this could be pushed into archiving but I don’t think that actually is practical - such concepts really need to be built deep into the core. I’m not advocating archiving-in-core here because I think archiving should like outside core. I do think however that there is value in the core having access to archive data to implement fine grained subscription control and I think it should do so on its own terms (and in its own database) rather than asking archivers to provide the needed functionality.
Fine grained subscription control whilst probably relatively involved to implement, is sure to have a huge payoff in giving users more relevance and less noise, perhaps increasing uptake of Mailman in situations where the noise and relevancy problem currently make Mailman impractical.
Thoughts anyone?
as
Quoting @barry ….
http://mailman.9.n7.nabble.com/Improving-the-archives-tt31296.html#a31340
“”"I like this for several reasons. I've long wanted a bridge between the traditional mailing list and a forum because to me they’re related along a spectrum of emotional investment.
What I mean is this. For the subjects and projects I care deeply about, I join the mailing list. I want to be intimately involved in the day-to-day collaboration that being subscribed gives me. I care enough about that that I'm willing to put up with the pain that comes along with mailing lists, such as the overhead for subscribing, deleting topics I don't care about, the occasional spam, the overhead of going on vacation or leaving the list, etc.
But there are even more topics or projects that I have only a fleeting interest in. Say I find a bug in some X program, or wake up and decide to learn how to use setuptools, or find that some recent update broke my Linux server. In all those cases, I might want to start a thread of discussion or ask a question, and be very involved in that thread for a week or two. Then, my interest wanes, or I get my question answered, or other projects pique my interest. Mailing lists are pretty bad at managing those kinds of fleeting involvement, but forums are quite nice. There's usually fairly low overhead (and probably even less if OpenID and such were in widespread adoption) for joining, and when I lose interest the forum doesn't fill up my inbox. OTOH, forums seem good for short 'instant' messages, but not so good (IMO) for free ranging, detailed discussions. So there's a spectrum. “””
@barry, @stephen, @terri
I know this is a wall of text (sorry) but if you have time I’d be interested to hear your thoughts as Dynamic Sublists are in the GSOC projects list for this year.
as
On 20 Mar 2015, at 9:53 am, Andrew Stuart <andrew.stuart@supercoders.com.au> wrote:
Refer to the bottom of this email for some previous quotes from @barry on this topic. I’ve also had off list discussions with Barry in which he has mentioned this same topic so it seems to have some previous thought gone into it.
I’m wanting to raise the topic of “fine grained subscription control” (for want of a better term) for discussion. Please note these thoughts refer to discussion style lists rather than notification style lists. Please prefix every sentence here with IMO - I’m not saying I’m right about anything, just putting forward food for thought. Also, I’m no Mailman expert so if my assumptions are plain wrong please let me know.
One of the core problems of mailing lists (at least as implemented in Mailman) is that there’s not much middle ground between being very involved (i.e receive all posts to list), or being almost-not-involved (i.e. receiving daily digests or no digest).
List noise and relevancy is the main problem and it’s a big impediment to lists being more widely used.
Very engaged users might be ok with constantly deleting messages that they are not interested in (i.e. irrelevant noise). Less engaged users will be annoyed with the noise and unsubscribe, or will switch to digest notification which I suggest to you is close to unsubscription anyway because that user no longer sees emails that they might have been interested in had they known there was a message on that subject. Digest users I suggest are at best observers rather than participators.
The more active the list, the greater the noise problem, the more users will drop out of the list. As a list gains more subscribers, eventually even the most engaged user will have had enough of the volume of messages and will drop back to digest or write some mailbox rule that moves the emails to a folder, effectively dis-engaging them from the list conversation flow. Thus Mailman discussion-style lists currently don’t scale.
Even for low volume lists, noise is a major inhibitor to usage in certain contexts. Consider for example the CIO of a company or the manager of a division of 30 developers. There are various mailing lists being used by the various projects that they are responsible for. The leader is not participating however because the noise from those lists would be overwhelming. They would however like to be partipcating in list discussions either that they initiated, are explicitly copied into, or relate to topics (keywords) that they are interested in. In reality, I’m not convinced Mailman would often be used in contexts like this currently because the relevancy/noise/digest/unsubscribe problem is a showstopper.
The systers have recognised this problem and their solution is Dynamic sublists as described here: http://wiki.list.org/DEV/Dynamic%20Sublists "List subscribers can decide whether to be a part of new conversations or not. If the users decide to subscribe to new conversations, then they will receive all the messages of a new conversation unless they explicitly unsubscribe from it. If they otherwise decide to unsubscribe from new conversations,then they will receive only the first message of every new conversation unless they explicitly subscribe to it.”
I don’t think dynamic sublists are an effective solution (IMO, remember?). Getting the root message in all conversations is still too noisy and its a clumsy mechanic to have to opt in to further discussion and my thinking is that opt-in-to-discuss will impose a barrier to engagement that will reduce user interaction - put another way - “opt in to every discussion? too hard”.
Possibly a solution worth considering is fine-grained subscription control, in which there is a set of SEND and DONTSEND rules at a List default level and at a user level.
SEND: Discussions where list member email address is in to/from/cc: Discussions where to/from/cc contains one or more of: sally@example.org, davina@example.org, *.example.com Discussions where subject contains keyword: hyperkitty Discussions where body contains keyword: hyperkitty
DONTSEND: Discussions where list member email address is in to/from/cc: Discussions where to/from/cc contains one or more of: sally@example.org, davina@example.org Discussions where subject contains keyword: Java Discussions where body contains keyword: ruby
The technical hurdle to making this work is that Mailman needs access to historical messages to make it work (i.e. integrating some level of aqrchive like functionality into the Mailman database). I suggest this this may not be as hard as it sounds and hey, we’ve got a database there anyway so why not use it?
One possibility is that this could be pushed into archiving but I don’t think that actually is practical - such concepts really need to be built deep into the core. I’m not advocating archiving-in-core here because I think archiving should like outside core. I do think however that there is value in the core having access to archive data to implement fine grained subscription control and I think it should do so on its own terms (and in its own database) rather than asking archivers to provide the needed functionality.
Fine grained subscription control whilst probably relatively involved to implement, is sure to have a huge payoff in giving users more relevance and less noise, perhaps increasing uptake of Mailman in situations where the noise and relevancy problem currently make Mailman impractical.
Thoughts anyone?
as
Quoting @barry ….
http://mailman.9.n7.nabble.com/Improving-the-archives-tt31296.html#a31340
“”"I like this for several reasons. I've long wanted a bridge between the traditional mailing list and a forum because to me they’re related along a spectrum of emotional investment.
What I mean is this. For the subjects and projects I care deeply about, I join the mailing list. I want to be intimately involved in the day-to-day collaboration that being subscribed gives me. I care enough about that that I'm willing to put up with the pain that comes along with mailing lists, such as the overhead for subscribing, deleting topics I don't care about, the occasional spam, the overhead of going on vacation or leaving the list, etc.
But there are even more topics or projects that I have only a fleeting interest in. Say I find a bug in some X program, or wake up and decide to learn how to use setuptools, or find that some recent update broke my Linux server. In all those cases, I might want to start a thread of discussion or ask a question, and be very involved in that thread for a week or two. Then, my interest wanes, or I get my question answered, or other projects pique my interest. Mailing lists are pretty bad at managing those kinds of fleeting involvement, but forums are quite nice. There's usually fairly low overhead (and probably even less if OpenID and such were in widespread adoption) for joining, and when I lose interest the forum doesn't fill up my inbox. OTOH, forums seem good for short 'instant' messages, but not so good (IMO) for free ranging, detailed discussions. So there's a spectrum. “””
On Mar 20, 2015, at 09:53 AM, Andrew Stuart wrote:
The systers have recognised this problem and their solution is Dynamic sublists
And we may have a GSoC on dlists. I think Systers is pretty happy with dlist functionality, but it'll be interesting to see how it works as a more general feature.
Related, there is the Roundup tracker's nosy lists.
http://roundup.sourceforge.net/docs/design.html?highlight=nosy#nosy-lists
Nosy lists are one of my favorite features of Roundup, which the Python project uses. The one problem with it is knowing that the new tracker issues are sent to a different mailing list, so until you subscribe to it, you may not be seeing new issues you want to be nosied on.
I see dlists and nosy lists as being somewhat related, and I'm hoping that any GSoC work that happens keeps an eye on commonalities that can be used later (though I don't want require nosy support of any GSoC student).
The other thought I've always had is to basically deliver a Gmane like experience with MM3. Gmane is one of the best FLOSS resources out there, but because it's an independent service, it doesn't always have the list mirror you care about. What if you could have a similar experience as soon as you installed Mailman? I don't know how prevalent NNTP clients are today; I know my MUA of choice (Claws Mail) has excellent support for it. So good in fact that I've tended to disable my subscriptions to many mailing lists I care about. I keep the subscription active so I can post to the list, but I read them almost exclusively through NNTP.
Possibly a solution worth considering is fine-grained subscription control, in which there is a set of SEND and DONTSEND rules at a List default level and at a user level.
SEND: Discussions where list member email address is in to/from/cc: Discussions where to/from/cc contains one or more of: sally@example.org, davina@example.org, *.example.com Discussions where subject contains keyword: hyperkitty Discussions where body contains keyword: hyperkitty
DONTSEND: Discussions where list member email address is in to/from/cc: Discussions where to/from/cc contains one or more of: sally@example.org, davina@example.org Discussions where subject contains keyword: Java Discussions where body contains keyword: ruby
This is similar to MM2's topics system. I'm not really sure how successful topics are. I tend to think they're too heavyweight to set up and use so most people and list admins don't bother.
The technical hurdle to making this work is that Mailman needs access to historical messages to make it work (i.e. integrating some level of aqrchive like functionality into the Mailman database). I suggest this this may not be as hard as it sounds and hey, we’ve got a database there anyway so why not use it?
Right. MM3 core has a message store, and I've always thought of that as a very low level raw historical record. It will never be as feature-full as a real archiver, but it could be used to record basic information such as message-ids, threading headers, etc.
Needless to say, this is an area I think is quite interesting and important to explore.
Cheers, -Barry
I wasn’t aware of MM2 topics. MM2 topics looks interesting but I agree its implementation is cumbersome. I think somewhere in the intersection of nosy lists, dlists, topics and fine-grained-subscription controls is an elegant way to reduce noise and improve relevance and user experience. I’m hoping to circle back to all this some months down the track, no time to do anything on it now.
as
participants (2)
-
Andrew Stuart -
Barry Warsaw