Mailman 3 March 2015 - Mailman-Developers

Re: [Mailman-Developers] SpamAssassin integration in mm3
by Mark Sapiro 10 Mar '15

10 Mar '15

On 03/09/2015 12:41 PM, Prakash kumar wrote: > > I want to work on the plugin for SpamAssassin integration in mm3. I have > installed SpamAssassin. > Do we need to add it as a rule in chains? > I saw that SpamAssassin's code is in perl, so what type of integration is > actually expected? I'm not a GSoC mentor, so my opinion is not authoritative, but this is much too small for a GSoC project. See <https://bugs.launchpad.net/mailman/+bug/266588> for a Mailman 2.1 handler that calls spamd to scan the mail and https://bugs.launchpad.net/mailman/+bug/558094> for a handler that looks ad headers added by SpamAssassin running ahead of Mailman. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 0

SpamAssassin integration in mm3
by Prakash kumar 10 Mar '15

10 Mar '15

Hello, I want to work on the plugin for SpamAssassin integration in mm3. I have installed SpamAssassin. Do we need to add it as a rule in chains? I saw that SpamAssassin's code is in perl, so what type of integration is actually expected? Thank You Prakash Kumar

1 0

Re: [Mailman-Developers] GSoC
by Vlad Shablinsky 09 Mar '15

09 Mar '15

> > Do you have something running on your `9001` port? This error means > mailman test server has problems binding to your local 9001 port as > evident from the error message. > > > 2. The second issue is that I get NameError when trying `dump_list` in > > `mailman shell` (I follow these instructions > > > http://gnu-mailman.readthedocs.org/en/latest/src/mailman/model/docs/address… > > ) > > > > First you need to import the `dump_list` function to be able to use it. > try: > > from mailman.testing.documentation import dump_list > > > ">>> dump_list(user.display_name for user in user_manager.users) > > Traceback (most recent call last): > > File "<console>", line 1, in <module> > > NameError: name 'dump_list' is not defined" > > > > > > 3. I've registed a user via Mozilla Persona and I'm able to log in, but I > > don't know what to do with it. I just have one field filled with email of > > my registred user. Everything is blank. Are the lists added using mailman > > shell? > > When you did `python manage.py syncdb` in postorius_standalone, did you > create a superuser? If yes, then use that email address to login and > you'd have lots of others options to play with. If not, then simply go > to postorius_standlaone directory and execute the command: > > python manage.py createsuperuser > > > I havent't started fixing a bug yet, but I suppose I have to use web UI > or > > the `mailman shell` to see if the bug is fixed or not. I'm supposed to be > > able to reproduce the state that causes the bug. I'm not sure that > > everything works fine since I have errors described above. Could you > guide > > me on that? > > The usual debugging process after fixing a bug varies from person to > person. There may be a lot of things that you won't be able to find in > Postorius. We generally use a combination of both shell and > mailman.client for our purposes. 1. When I start tox I have sudo lsof -i tcp:9001 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME python3.4 12874 vlad 22u IPv4 0x1f87e35d769a88e9 0t0 TCP localhost:etlservicemgr (LISTEN) then when an the error occurs I have Traceback (most recent call last): File "/Users/vlad/Projects/GSoC/Mailman/mailman/.tox/py34/bin/runner", line 9, in <module> load_entry_point('mailman==3.0.0b6', 'console_scripts', 'runner')() File "/Users/vlad/Projects/GSoC/Mailman/mailman/src/mailman/bin/runner.py", line 200, in main runner = make_runner(*args.runner, once=args.once) File "/Users/vlad/Projects/GSoC/Mailman/mailman/src/mailman/bin/runner.py", line 100, in make_runner return runner_class(name, slice) File "/Users/vlad/Projects/GSoC/Mailman/mailman/src/mailman/runners/lmtp.py", line 164, in __init__ smtpd.SMTPServer.__init__(self, localaddr, remoteaddr=None) File "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/smtpd.py", line 589, in __init__ self.bind(localaddr) File "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncore.py", line 334, in bind return self.socket.bind(addr) OSError: [Errno 48] Address already in use and when I look for processes: sudo lsof -i tcp:9001 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME python3.4 12874 vlad 22u IPv4 0x1f87e35d769a88e9 0t0 TCP localhost:etlservicemgr (LISTEN) python3.4 19844 vlad 23u IPv4 0x1f87e35d71e828e9 0t0 TCP localhost:56728->localhost:etlservicemgr (SYN_SENT) when some time passes I have this in my terminal: Traceback (most recent call last): File "/Users/vlad/Projects/GSoC/Mailman/mailman/.tox/py34/bin/runner", line 9, in <module> load_entry_point('mailman==3.0.0b6', 'console_scripts', 'runner')() File "/Users/vlad/Projects/GSoC/Mailman/mailman/src/mailman/bin/runner.py", line 200, in main runner = make_runner(*args.runner, once=args.once) File "/Users/vlad/Projects/GSoC/Mailman/mailman/src/mailman/bin/runner.py", line 100, in make_runner return runner_class(name, slice) File "/Users/vlad/Projects/GSoC/Mailman/mailman/src/mailman/runners/rest.py", line 55, in __init__ self._server = make_server() File "/Users/vlad/Projects/GSoC/Mailman/mailman/src/mailman/rest/wsgiapp.py", line 163, in make_server handler_class=AdminWebServiceWSGIRequestHandler) File "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/wsgiref/simple_server.py", line 153, in make_server server = server_class((host, port), handler_class) File "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", line 429, in __init__ self.server_bind() File "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/wsgiref/simple_server.py", line 50, in server_bind HTTPServer.server_bind(self) File "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/server.py", line 133, in server_bind socketserver.TCPServer.server_bind(self) File "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", line 440, in server_bind self.socket.bind(self.server_address) OSError: [Errno 48] Address already in use Exception in thread Thread-2: Traceback (most recent call last): File "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/threading.py", line 921, in _bootstrap_inner self.run() File "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/threading.py", line 869, in run self._target(*self._args, **self._kwargs) File "/Users/vlad/Projects/GSoC/Mailman/mailman/src/mailman/testing/helpers.py", line 215, in loop self.start_check() File "/Users/vlad/Projects/GSoC/Mailman/mailman/src/mailman/rest/tests/test_membership.py", line 223, in _wait_for_both wait_for_webservice() File "/Users/vlad/Projects/GSoC/Mailman/mailman/src/mailman/testing/helpers.py", line 285, in wait_for_webservice int(config.webservice.port))) TimeoutError: [Errno 60] Operation timed out and sudo lsof -i tcp:9001 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME python3.4 12874 vlad 22u IPv4 0x1f87e35d769a88e9 0t0 TCP localhost:etlservicemgr (LISTEN) 2. I imported dump_list like you said and It started to work. Now I can't create list >>> mlist = create_list('test(a)example.com') Traceback (most recent call last): File "<console>", line 1, in <module> File "/Users/vlad/Projects/GSoC/Mailman/mailman/src/mailman/app/lifecycle.py", line 74, in create_list raise BadDomainSpecificationError(domain) mailman.interfaces.domain.BadDomainSpecificationError: example.com I registred a superuser in postorius_standalone. Now I can create a domain and lists. What do I have to write in the forms to test it somehow? Thanks, Vlad Shablinsky

1 0

Re: [Mailman-Developers] Discussion about Mailman plugins(GSOC 2015)
by Stephen J. Turnbull 09 Mar '15

09 Mar '15

Prakash kumar writes: > 1. Using regex for filtering texts ( personal information eg: phone > number, address ). What do you mean by "filter"? Stop or hold delivery, or "sanitize" the message? If the former, I would drop the motivation about "personal" information, because body filtering is also desired for spam-filtering and avoiding flame-war reasons. > 2. Checking type of files that can be attached to the mails. For eg: > .exe not allowed. This is right out as it's already implemented. > 3. If there are multiple attachments in the email > > If (multiple attachments) then > > for each attachment > > if(not appropriate) then > > discard > > notify sender that this part is removed from email body and why > > else > > continue Also already done (except for the notification part. > Is it big enough for a gsoc proposal? I don't think writing multiple unrelated plugins is a good proposal. AFAIK GSoC is intended to be a single project with enough scope to require a certain amoung of design and planning.

2 1

Re: [Mailman-Developers] Mailman installation error: GSOC 2015
by Mark Sapiro 09 Mar '15

09 Mar '15

On 03/08/2015 10:58 AM, NILMADHAB MONDAL wrote: > I got following error > E: Unable to locate package python-devel > E: Unable to locate package python3-devel > E: Unable to locate package python3-virtualen The packages are probably named python-dev, python3-dev and python3-virtualenv. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 0

Re: [Mailman-Developers] Approach for Auto moderation system
by Aanand Shekhar Roy 09 Mar '15

09 Mar '15

> One concern here is that "Thread" is a fragile term in email. Unless you > are planning on some form of message body analysis to group messages > together, you are going to need to rely on the In-Reply-To and > References headers of the incoming email, which can have its > difficulties. If you are going to thread by something else, like the > subject, you may find people making minor changes in the subject to > bypass the moderation. Yes, I agree ,but having a system that curbs this problem will not again be a very good solution, for eg. A thread regarding discussion on mailman2 can have the name of the thread "Mailman 2", and for mailman3, the thread will be named as "Mailman 3". Now ,it can also appear that someone has made a minor change to avoid moderation but it is not so. > > First, these headers are optional, and some mail agents may not generate > them, and more importantly, the subscriber can bypass this linking by > creating a reply as a "new message" thus bypassing the auto moderation. Since I wish to implement this system as a plug-in it will be optional for the list admins and having an MTA that generates headers can be kept as a requirement. > Second, there is an unreliability in these headers as they will not > necessarily reference the "start" of the thread, but may only list > messages later in the thread, and to get your "Thread Name", you are > going to have to keep a full history (for some period back) of messages > and what thread you determined them to be in to figure out what thread > this message is in. What we can do is to remove general keywords like "Re:[ ]" and "Fwd:[ ]" from the thread and then add to the database, so we don't need to store all the history, just information about last one does the job, we check that through Table1. > This means that any system that ties to limit the rate "in thread" must > also have a similar (but perhaps different value) limit on total > postings or creation of new threads. We do not aim at limiting threads or posts on a thread we are just slowing it down to provide room to other users. If we decrease no of posts on a thread it might affect the discussion going on as some threads are very long and also important and can't be shortened. Regards Aanand

4 4

Re: [Mailman-Developers] Query regarding plugin integration with mailman3
by Aanand Shekhar Roy 09 Mar '15

09 Mar '15

I am extremely sorry for behaving stupidly. I in no means meant to insult you. You are far senior in all terms and I am ashamed to have dis-honoured you unwillingly. I got a bit excited over that without thinking rationally. I'll try my best not repeat such an illogical action in future. I'll work for a better proposal for gsoc. Sorry again. Regards Aanand

2 1

Approach for Auto moderation system
by Aanand Shekhar Roy 09 Mar '15

09 Mar '15

Hi, Does the following approach for auto moderation sound fine or am I on the wrong path ? I think we neet to make changes in handler to_outgoing.py in method process. 1) We need to make two databases : Table1 Schema-> [ user_id | Thread name | list_id | count | last_msg_posted_at ] user_id, list_id, Thread name together act as primary key. It stores the count of number of posts of a user on a particular thread and the time corresponding to the last post. Table2 Schema-> [message_id | user_id |Thread name| list_id |don't_post_before_time] here, don't_post_before_time=(last_msg_posted_at)+(count/10)*d where, "d" will be delay set by list-admin, eg 3 hrs. 2) The count variable is accessed from Table1. 3) When process is called and a new message entry comes: a) case 1: It is the first post by user on that thread then we create a new entry in Table1 with count 1 and last_msg_posted_at_time = posting time of that message. Then the corresponding entry is made in Table2 with new count and calculated don't_post_before_time. case 2: It is not the first post implies there is already an entry in Table1 then we increase its count by 1 and we calculate don't_post_before_time using count and last_msg_posted_at_time. After making the corresponding entry in Table2 using values just calculated we update last_msg_posted_at_time in Table1 with the value of don't_post_before_time of this entry b)Now we iterate over all entries in Table2 and whoever has don't_post_before_time < current_time we remove the corresponding entry form Table2 and enqueue it in outgoing queue else it stays in the database. Suppose, "abc(a)gmail.com" sends 9 messages to a list "foo(a)foo.com" on the thread "Questions",the last being sent at 4:00pm, then Table1 will look like [abc(a)gmail.com | Questions| foo(a)foo.com | 9 | 4:00pm] Now, if the 10th msg comes,from this database, we will retrieve the "last_msg_posted_at" before updating the tuple for the 10th msg and will calculate dont_post_it_before=last_msg_posted_at+(count/10)*d = 7:00pm, now the last_msg_posted_at field in the first database will be updated and will be equal to dont_post_it_before, and these details will be used to create an entry in Table2 database too. Table1 will look like [abc(a)gmail.com | Questions| foo(a)foo.com | 10 | 7:00pm] Table2 will look like [xyz | abc(a)gmail.com | Questions| foo(a)foo.com | 7:00pm] Now we iterate over all entries and enqueue all that have don't_post_before_time < current_time. Remaining entries stay in the database and will be checked the next time process is called. Regards Aanand

3 2

Feature Review : Mass Removal
by Bhavesh Goyal 09 Mar '15

09 Mar '15

Hello people, Similar to the Mass Subscribe option I was wondering if we need a mass removal option too. I saw the idea of having such feature in one of the bug reports and thought it would be quite useful to have one such feature implemented in postorius. It may be useful for a large list where list owner doesn't want to delete all the manually one by one neither does he wants to go through the process of configuring up a new list. If it is actually something good to have integrated in mailman based on everyone's opinion, i would like to get started on implementing it too.. :) -- Regards, Bhavesh Goyal

1 0

GSOC, Anonymous lists
by Pavan Koli 09 Mar '15

09 Mar '15

My name is Pavan Koli I'm interested in the project idea(Anonymous lists). My rough idea of implementing it is using modifications in the schema of the database table. Just before sending of messages in the From header actual mail id will be replaced by the anonymous id. Reply-To to be the anonymous id of the sender. This is just the rough idea, I'm yet to think about the entire implementation and approach. For that please point me in the right direction and documentation.

1 0