Advanced user reputation/moderation features
![](https://secure.gravatar.com/avatar/5a4f12c3cd289097832e8cc7e0c31073.jpg?s=120&d=mm&r=g)
Let me state that I am not a IT person, so some of my questions/comments may not be the most appropriate lingo that is used by you folks. It is also very likely that my issues/ideas have been raised before, though I have not been able to find it in the archives. I have been moderating non-commercial mailing lists (initially majordomo and then mailman) since the mid 1990s.
One of the biggest issues has been in the moderation of posts on these discussion lists. Usually, we have had a set of volunteer moderators who review the posts for spam, obscenity, personal attacks etc before they get approved for distribution. While stuff like spam, obscenity etc are for the most part obvious and can be auto filtered using various options, the problem remains that the issue of more mundane posts are still subject to the moderator's bias. Besides being subjective, it can be many, many hours before a post gets approved, which hampers the continuity and spontaneity of the posts.
An alternate approach to moderation could be via a "user reputation score" as is often found in many web based discussion forums. A person with a higher user reputation score could have fewer restrictions posed on him, in terms of the number of posts (say per unit of time) without moderation, before he gets flagged for manual moderation. At this stage, it is not my intent to propose an a specific mathematical algorithm, but rather to bring out general concepts.
The issue of course is how would a "user reputation score" be determined? I would think it would be composed of a variety of components such as: (in no particular order)
Feedback scores as determined by the general mailing list population. The feature could be embedded into an email to allow collective input from the reader base. This will automatically provide a feedback loop to poor quality posters to either reign in their posts or suffer a demotion in their posting rights via the pre-defined algorithm. Furthermore, it will be based on the fopinions of the collective masses, rather than that of the moderator.
Verified identity/open ID: I understand that open ID is one of the features being considered for a future release of mailman. Individuals with larger networks on sites such as facebook etc could be given a higher starting reputation score as opposed to a ID that has little or no prior history or network to back that person's identity. This will help reduce the possibility of fraudulent posts under freshly created IDs.
I would appreciate some feedback on whether something like this is possible. Since I am not a programmer, I would not be able to provide technical support for the development of this code. However, I may be able to get some funding from a non-profit foundation to get this done.
Regards, Marlon
![](https://secure.gravatar.com/avatar/de4632b78ba00436a9b77ed0d6ea8877.jpg?s=120&d=mm&r=g)
On Tue, Jan 19, 2010 at 05:42:13PM -0600, Marlon Menezes wrote:
Individuals with larger networks on sites such as facebook etc could be given a higher starting reputation score as opposed to a ID that has little or no prior history or network to back that person's identity. This will help reduce the possibility of fraudulent posts under freshly created IDs.
The "numbers game" concerns me slightly; I don't believe a large amount of "friends" necessarily equates to a greater level of trust.
Perhaps this may best be highlighted with 'spammers' on Twitter, for example (those following a large amount of people, compared with a representatively small amount of followers); or, indeed, the myspace "get as many 'friends' as you can" game.
Not much help, more of a dampener on things.
On the reverse, if one were to use (G|P)GP keys, and the amounts of verifications/web-of-trust there: a lot of us are really quite lazy when it comes to key-signing.
About the only site with a trustworthy reputation, I would suggest, would have been LinkedIn, although, I'm not too sure I'd even stand-by that these days, as it's changed rather a lot since when *I* signed up to that.
Were there to be a scheme, I think *I* would tend to go with something operating per-list, in the stack-overflow approach you're suggesting; were something that complex to be introduced; I'm lazy, and think if someone's wrong, they'll be shouted down; when people can 'prove' they're not barking mad, they're set as unmoderated.
That said, for a majority of the lists I listmaster, it's a "you can post to the list, once you join, until you annoy the list(master)".
And annoying the listmaster usually means spamming/posting out-of-office messages to the list, or being *exceptionally* out of tune with the rest of the list.
(With MM3, I could see, perhaps, this maybe being useful as an optional plugin, mind.)
-- ``Of course we are not patronising women. We are just going to explain to them, in words of one syllable, what it is all about.'' (Olga Maitland)
![](https://secure.gravatar.com/avatar/5a4f12c3cd289097832e8cc7e0c31073.jpg?s=120&d=mm&r=g)
On Tue, Jan 19, 2010 at 6:41 PM, Adam McGreggor <adam-mailman@amyl.org.uk>wrote:
Individuals with larger networks on sites such as facebook etc could be given a higher starting reputation score as opposed to a ID that has little or no prior history or network to back that person's identity. This will help reduce
On Tue, Jan 19, 2010 at 05:42:13PM -0600, Marlon Menezes wrote: the
possibility of fraudulent posts under freshly created IDs.
The "numbers game" concerns me slightly; I don't believe a large amount of "friends" necessarily equates to a greater level of trust.
Perhaps this may best be highlighted with 'spammers' on Twitter, for example (those following a large amount of people, compared with a representatively small amount of followers); or, indeed, the myspace "get as many 'friends' as you can" game.
I think one way to get around this would be to give higher value to friends who are already on the mailing list. The real issue I am trying to address here is not identity verification, but rather, reducing the chance of a potential abuser who has been put in the moderated list, then tries to get in un-moderated under a new ID. With such a set up, it will require a lot of effort on the part of that user to generate a new friends list etc.
That said, for a majority of the lists I listmaster, it's a "you can post to the list, once you join, until you annoy the list(master)".
This approach, while being suitable for a low volume technical group such as this, does not work for a highly active community/social forum in which what defines what is annoying the list master is subject to interpretation.
Regards, Marlon
![](https://secure.gravatar.com/avatar/173371753ea2206b9934a9be1bdce423.jpg?s=120&d=mm&r=g)
On Jan 19, 2010, at 05:42 PM, Marlon Menezes wrote:
An alternate approach to moderation could be via a "user reputation score" as is often found in many web based discussion forums. A person with a higher user reputation score could have fewer restrictions posed on him, in terms of the number of posts (say per unit of time) without moderation, before he gets flagged for manual moderation. At this stage, it is not my intent to propose an a specific mathematical algorithm, but rather to bring out general concepts.
First I'm going to explain how we do something similar in Launchpad[1], then I can talk about how we might go about this in Mailman 3.
In Launchpad, we do several things to reduce the moderation burden while not opening up lists to too much spam. One advantage we have in Launchpad is that we accept no email from unregistered or unvalidated email addresses. Now, this is site-wide so once you've registered and validated your email address to Launchpad, then you can potentially email any mailing list, even ones you are not a member of. Before that, we summarily reject your message. The validation step is of course important, because once you're registered with Launchpad you can add more email addresses, but you can't really use them until they've been validated via mail-back confirmation.
Okay, that's the first step, and it can do a pretty good job of reducing crap. Next, if you are a member of the mailing list, we allow your message to go through. Because list moderators have tools at their disposal to punish abusers we think this is generally a fine trade-off. I'm less concerned about allowing one bad message if you can boot the offender off your list or ban them permanently if they persist. IOW, I'd rather punish the rare bad apple than impose additional burdens on the vast majority of good ones.
Let's say you post to a mailing list that you are not a member of, what happens? Well, the first time you post to it, your message gets held for moderator approval. This is very similar to what happens in Mailman today. The difference is that if the moderator approves your post, you then your subsequent messages to that mailing list are not moderated. The numbers can be tweaked, but the idea is the same: if you prove yourself by posting some good messages a few times, we don't need to moderate you any more.
Taking this concept further, if you post to three different mailing lists that you are not a member of, and your held message gets approved each time with no rejections, we think you're probably okay in general and will let you post to any mailing list without moderation, regardless of whether you're a member or not. See, your reputation is improving so you get more privileges.
Each user in Launchpad also has a concept of "standing", which the admins can set but users cannot see or influence. There are essentially four levels of standing: excellent, good, unknown, and poor. If your standing is poor, we summarily discard your messages. If your standing is good or better, then your posts go through without moderation. So one of the tools we have to punish abusers is to set their standing to "poor" and wave goodbye.
Now, even though Launchpad is backed by Mailman 2, most of the data for this is in Launchpad, so stock Mailman 2 can't really do this. The main reason is that Mailman 2 doesn't have any global concept of a user. It's a complete silo per mailing list. Mailman 3 fixes this architectural limitation so we could potentially record the same standing data and posting history in Mailman 3 and use very similar posting rules. I think this would help sites that run many mailing lists (e.g. python.org).
I could even see other moderation rules implemented as plugins. Mailman 3 has a more robust architecture, with separate pipelines for moderation and modification of messages. The rule chains that handle moderation could easily be extended to look for any of the above criteria, as well as any other user reputation scores you can dream up.
This all works great in theory, with of course the fundamental problem of email: without digital signatures, it's completely spoofable. Every byte of data that Mailman can possibly look at can be forged, so it could make a completely reasonable decision based on false data, and then you're pretty much SoL. In practice, this doesn't seem to be a big problem, but really paranoid sites with knowledgeable users could require that all messages be digitally signed in order to even be considered for acceptance. Personally, I'd love to see that become the normal across the board, but I'm not holding my breath. ;)
I hope that helps in understanding where my current thinking on the subject is. While the framework for implementing the above exists and works in the current Mailman 3 alphas, the actual reputation code isn't there. That's something that focused (and funded?) development work could push forward.
-Barry
![](https://secure.gravatar.com/avatar/2dea2c1eb1af99c64c1d3c84769eb314.jpg?s=120&d=mm&r=g)
Hi Barry, Keep up the good work! In response to your last email:
With the "global user" concept, would the individual list moderators be able to assign a "poor" status for one list only? Or will all the lists be stuck with the status assigned on a global basis?
vince heuser
![](https://secure.gravatar.com/avatar/173371753ea2206b9934a9be1bdce423.jpg?s=120&d=mm&r=g)
On Jan 21, 2010, at 11:14 AM, vince@wlcr.net wrote:
Hi Barry, Keep up the good work! In response to your last email:
Thanks!
With the "global user" concept, would the individual list moderators be able to assign a "poor" status for one list only? Or will all the lists be stuck with the status assigned on a global basis?
In Launchpad, standing is an attribute of the user, so it is global across all mailing lists. I think that makes the most sense, since I don't think a user in poor standing on list A is going to be much better on list B. However list admins do have other means to control who can post to their list, so they'll have other ways to manage that on a per-list basis.
Cheers, -Barry
![](https://secure.gravatar.com/avatar/5a4f12c3cd289097832e8cc7e0c31073.jpg?s=120&d=mm&r=g)
On Wed, Jan 20, 2010 at 4:51 PM, Barry Warsaw <barry@list.org> wrote:
Taking this concept further, if you post to three different mailing lists that you are not a member of, and your held message gets approved each time with no rejections, we think you're probably okay in general and will let you post to any mailing list without moderation, regardless of whether you're a member or not. See, your reputation is improving so you get more privileges.
Each user in Launchpad also has a concept of "standing", which the admins can set but users cannot see or influence. There are essentially four levels of standing: excellent, good, unknown, and poor. If your standing is poor, we summarily discard your messages. If your standing is good or better, then your posts go through without moderation. So one of the tools we have to punish abusers is to set their standing to "poor" and wave goodbye.
Barry,
Thanks for your detailed explanation. The concept of "standing" is in line with my thinking, but I would like to extend it even further. As of now, one's standing is completely in the hands of the list administrator, which again, may be fine for a low volume technical mailing list. Why not democratize it by allowing for input/feedback from the list members? Right now, one of the biggest issues we are facing is that in very high volume community/opinion discussion lists, the list administrators need to scan hundreds of posts per day from legitimate members. Discussions often get heated and discussion threads can go on endlessly. Sometimes, administrators, may take actions that may not be appropriate, given their inability to keep up with the discussions. If the quality scores of posts and discussion threads could be farmed out to the general list, it would drastically reduce the burden on the administrators.
The issue of identify verification, spoofing etc are valid, but those instances are exactly the occasions where the list admins should have time to focus on.
Regards, Marlon
participants (4)
-
Adam McGreggor
-
Barry Warsaw
-
Marlon Menezes
-
vince@wlcr.net