Approach for Auto moderation system
![](https://secure.gravatar.com/avatar/869b79cce5d993c2729bc12c7195aa19.jpg?s=120&d=mm&r=g)
Hi, Does the following approach for auto moderation sound fine or am I on the wrong path ?
I think we neet to make changes in handler to_outgoing.py in method process. form Table2 and enqueue it in outgoing queue else it stays in the
We need to make two databases : Table1 Schema-> [ user_id | Thread name | list_id | count | last_msg_posted_at ] user_id, list_id, Thread name together act as primary key. It stores the count of number of posts of a user on a particular thread and the time corresponding to the last post. Table2 Schema-> [message_id | user_id |Thread name| list_id |don't_post_before_time] here, don't_post_before_time=(last_msg_posted_at)+(count/10)*d where, "d" will be delay set by list-admin, eg 3 hrs.
The count variable is accessed from Table1.
When process is called and a new message entry comes: a) case 1: It is the first post by user on that thread then we create a new entry in Table1 with count 1 and last_msg_posted_at_time = posting time of that message. Then the corresponding entry is made in Table2 with new count and calculated don't_post_before_time. case 2: It is not the first post implies there is already an entry in Table1 then we increase its count by 1 and we calculate don't_post_before_time using count and last_msg_posted_at_time. After making the corresponding entry in Table2 using values just calculated we update last_msg_posted_at_time in Table1 with the value of don't_post_before_time of this entry b)Now we iterate over all entries in Table2 and whoever has don't_post_before_time < current_time we remove the corresponding entry
database.
Suppose, "abc@gmail.com" sends 9 messages to a list "foo@foo.com" on the thread "Questions",the last being sent at 4:00pm, then Table1 will look like [abc@gmail.com | Questions| foo@foo.com | 9 | 4:00pm] Now, if the 10th msg comes,from this database, we will retrieve the "last_msg_posted_at" before updating the tuple for the 10th msg and will calculate dont_post_it_before=last_msg_posted_at+(count/10)*d = 7:00pm, now the last_msg_posted_at field in the first database will be updated and will be equal to dont_post_it_before, and these details will be used to create an entry in Table2 database too. Table1 will look like [abc@gmail.com | Questions| foo@foo.com | 10 | 7:00pm] Table2 will look like [xyz | abc@gmail.com | Questions| foo@foo.com | 7:00pm] Now we iterate over all entries and enqueue all that have don't_post_before_time < current_time. Remaining entries stay in the database and will be checked the next time process is called.
Regards Aanand
![](https://secure.gravatar.com/avatar/2d8b084fbf3bb480d8a3b6233b498f4f.jpg?s=120&d=mm&r=g)
On 3/7/15 10:27 AM, Aanand Shekhar Roy wrote:
Hi, Does the following approach for auto moderation sound fine or am I on the wrong path ?
I think we neet to make changes in handler to_outgoing.py in method process.
- We need to make two databases : Table1 Schema-> [ user_id | Thread name | list_id | count | last_msg_posted_at ] user_id, list_id, Thread name together act as primary key. It stores the count of number of posts of a user on a particular thread and the time corresponding to the last post. Regards Aanand
One concern here is that "Thread" is a fragile term in email. Unless you are planning on some form of message body analysis to group messages together, you are going to need to rely on the In-Reply-To and References headers of the incoming email, which can have its difficulties. If you are going to thread by something else, like the subject, you may find people making minor changes in the subject to bypass the moderation.
First, these headers are optional, and some mail agents may not generate them, and more importantly, the subscriber can bypass this linking by creating a reply as a "new message" thus bypassing the auto moderation.
Second, there is an unreliability in these headers as they will not necessarily reference the "start" of the thread, but may only list messages later in the thread, and to get your "Thread Name", you are going to have to keep a full history (for some period back) of messages and what thread you determined them to be in to figure out what thread this message is in.
This means that any system that ties to limit the rate "in thread" must also have a similar (but perhaps different value) limit on total postings or creation of new threads.
Note this is VERY different then the problem with a message board, where there is a much stronger definition of "Thread" and people may be much less inclined to "break thread" to bypass limits.
-- Richard Damon
![](https://secure.gravatar.com/avatar/334b870d5b26878a79b2dc4cfcc500bc.jpg?s=120&d=mm&r=g)
Aanand Shekhar Roy writes:
Hi, Does the following approach for auto moderation sound fine or am I on the wrong path ?
I wouldn't call it "automoderation", I would prefer "post throttling".
This is more suggestive of what is actually happening (moderation usually involves filtering on content).
I think we neet to make changes in handler to_outgoing.py in method process.
- We need to make two databases : Table1 Schema-> [ user_id | Thread name | list_id | count | last_msg_posted_at ] user_id, list_id, Thread name together act as primary key. It stores the count of number of posts of a user on a particular thread and the time corresponding to the last post.
I think you should provide an option to just throttle the user on a given list, too. Possibly with different default throttling parameters.
Table2 Schema->
I don't see why you would have a separate table for messages. Just add the post_after_time attribute to the message's metadata.
[message_id | user_id |Thread name| list_id |don't_post_before_time] here, don't_post_before_time=(last_msg_posted_at)+(count/10)*d
I think you probably want count//10 here IIUC. I'm guessing you want the delay to be zero until the poster hits 10, right? In Python 2, 5/10 == 0, but in Python 3 it's 0.5. 5//10 == 0.
where, "d" will be delay set by list-admin, eg 3 hrs.
The count variable is accessed from Table1.
When process is called and a new message entry comes: a) case 1: It is the first post by user on that thread then we create a new entry in Table1 with count 1 and last_msg_posted_at_time = posting time of that message. Then the corresponding entry is made in Table2 with new count and calculated don't_post_before _time. case 2: It is not the first post implies there is already an entry in Table1 then we increase its count by 1 and we calculate don't_post_before_time using count and last_msg_posted_at_time. After making the corresponding entry in Table2 using values just calculated we update last_msg_posted_at_time in Table1 with the value of don't_post_before_time of this entry
b)Now we iterate over all entries in Table2 and whoever has don't_post_before_time < current_time we remove the corresponding entry form Table2 and enqueue it in outgoing queue else it stays in the database.
In my proposal to put it in metadata, you would just enqueue the message in outgoing and have OutgoingRunner check for post_after_time. If not present or present and expired, post the message, otherwise leave it in queue. If you do it your way, you need to have a separate queue (storage) for messages waiting for their post_after_times.
participants (3)
-
Aanand Shekhar Roy
-
Richard Damon
-
Stephen J. Turnbull