Mailman 3 March 2015 - Mailman-Developers

Re: [Mailman-Developers] Query regarding ambiguous behavior of REST API due to case-sensitivity of email addresses
by Barry Warsaw 09 Mar '15

09 Mar '15

On Mar 03, 2015, at 02:15 PM, Ankush Sharma wrote: >> 2. Later on, in the same function the following code checks if the email >> is matched: >> >> *for address in user.addresses:* >> * if address.email == email:* >> * break* >> >> Here *email* is in *preserved case* while *address.email *returns a lower >> case version of email and an *unmatch* occurs. ( sam(a)gmail.com and >> SAM(a)gmail.com do not matches ofcourse ) I think you've done a great job of finding the root cause of the bug. I'll comment further on the merge proposal, but I think your fix is essentially correct. Thanks! Cheers, -Barry

1 0

Re: [Mailman-Developers] Mailman installation error: GSOC 2015
by NILMADHAB MONDAL 08 Mar '15

08 Mar '15

I got following error E: Unable to locate package python-devel E: Unable to locate package python3-devel E: Unable to locate package python3-virtualen when I tried to execute the command sudo apt-get install python-setuptools python-devel python-virtualenv python3-devel python3-virtualenv git bzr gcc nano I cannot python3 virtual env ... Please help On Sat, Mar 7, 2015 at 2:32 PM, Rajiv Sharma < rajiv.tilakraj.sharma(a)gmail.com> wrote: > Sorry abhilash, next time I will make sure I include the stack trace along > with the commands. Yes, setuptools.py is installed in my virtual > environment and I specified the python3.4 path with the -p argument while > creating the virtual env to install mailman. > > Turns out my virtualenv had setuptools for python 2.7 and hence I was > getting module already installed message on running apt-get or pip install. > The solution to this is either: > Sudo apt-get install python3-setuptools > Or: > Sudo pip install setuptools-3.4.py (or something similar mentioning the > version of setup tools) > > Hope this helps others. > > Thank you > Rajiv Sharma > -- Regards, Nil madhab Mondal Web team Head, Kshitij Technology Students Gymkhana Indian Institute of Technology, Kharagpur

1 0

Fwd: Anatomy of list ids does not keep with that of urls causes some REST end points to return 404 always
by Ankush Sharma 08 Mar '15

08 Mar '15

---------- Forwarded message ---------- From: Ankush Sharma <ankush.sharma.ece12(a)iitbhu.ac.in> Date: Sun, Mar 8, 2015 at 3:49 PM Subject: Re: [Mailman-Developers] Anatomy of list ids does not keep with that of urls causes some REST end points to return 404 always To: Richard Damon <Richard(a)damon-family.org> Well, thanks Richard for replying. On Sat, Mar 7, 2015 at 8:04 PM, Richard Damon <Richard(a)damon-family.org> wrote: > On 3/7/15 5:27 AM, Ankush Sharma wrote: > >> Hello everyone, >> >> I want to discuss a serious issue regarding mailman architecture in >> context >> with bug #1429366 <https://bugs.launchpad.net/mailman/+bug/1429366>. >> The *hash(#) >> *is a valid character as far as the local part of the email addresses is >> concerned as per the email RFCs. So, as the mailing list addresses are >> email addresses too, we can use # in the list names too. And, in context >> with mailman it works well. We can create a list with list_id >> *sam#hashed.host.org >> <http://hashed.host.org>* for the address *sam#hashed(a)host.org >> <hashed(a)host.org> *. This works fine. But it makes the *list_id *to >> contain >> the hash character and therefore the REST endpoint for retrieving list >> wise >> info becomes invalid, i.e : >> >> *<api-root>/lists/sam#hashed.host.org <http://hashed.host.org>* >> >> Because in an URL the stuff after # is treated as *document starting >> point* >> i.e an id identifier or something of a dom element. This is not a valid >> PATH for the server. Therefore the *falcon wsgi request object* does not >> contain information of that and the *req.path* simply returns *sam* as the >> *list_id* ( http://bazaar.launchpad.net/~mailman-coders/mailman/3.0/ >> view/head:/src/mailman/rest/wsgiapp.py#L65 >> ) giving a 404 because there is no any list with list id *sam*. >> The mailman client works fine, it sends a GET to >> *<api-root>lists/sam#hashed.host.org >> <http://hashed.host.org>. * >> >> This causes the REST end points which needs *list_id* to return 404 or in >> worse we can have a list_id clash between ids* sam#XXXXX* and *sam*. >> Further more if the list_id starts with a # character then the server >> finds >> list_id to be *empty string* and therefore we get a KEY ERROR because >> *fqdn_listname >> *is not set too. The bug highly effects postorius too. The lists index >> template at */postorius/lists/ *cannot be rendered as it uses the former >> REST endpoint and again a 404 is given. And, until we delete this list >> from >> the database, we can't do anything except of getting a 404 and KEY ERROR >> each time. >> As far as security is concerned, if an another user created a public list >> using a hash character, then public list indexing would also fail. >> >> *Possible fixes:* >> 1. Put a constraint over the use of this kind of characters, but it's like >> falling back. >> 2. Do not derive list ids directly from the list addresses, use some kind >> of hashing or encoding technique to generate list_ids. Need to change a >> lot >> in architecture; :( . >> 3. Use obscure techniques to retain the hash part as discussed here >> http://stackoverflow.com/questions/317760/how-to-get- >> url-hash-from-server-side >> . >> >> I think this is quite a serious issue in mailman architecture; i.e >> obtaining list_ids directly from list addresses. I have been trying to fix >> this bug using some obscure techniques. But as it is related to mailman >> architecture, I thought it should be worthy to discuss on the mailing list >> first. >> >> Thanks, >> Ankush Sharma >> github.com/black-perl >> >> The simple, and standard complient, solution would seem to be to define > that special characters get URL encoded as needed (and decoded on the > receiving end. I had a feeling of using URL encoding universally for all the URL building routines but the issue I faced is of identifying the set of URL building routines that actually require to encode URLs to prevent chaos. That's why I didn't listed in on the possible fixes list. Because in that case it would be difficult to identify these guys with full confidence. > It should be standard procedure that ANY routine building a URL would use > URL encoding on the pieces unless it is positive that it is not needed. But now are things more clear. I will try to keep a positive approach towards all these URL building routines keeping them in the suspected list till I am not sure about one. (URL encoding is replacing the problem chars with %nn where nn is the hex value of the character, this now makes % a problem char so it is sent as part of the URL as %25) > > -- > Richard Damon > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers(a)python.org > https://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/ > mailman-developers%40python.org/ > Unsubscribe: https://mail.python.org/mailman/options/mailman- > developers/ankush.sharma.ece12%40itbhu.ac.in > > Security Policy: http://wiki.list.org/x/QIA9 > Can it be a security concern as if any other user created such a public list then public listing would simply fail ? I haven't tried reproducing this part on my machine yet. But I am trying now. Thanks, Ankush Sharma github.com/black-perl

1 0

Re: [Mailman-Developers] Discussion about Mailman plugins(GSOC 2015)
by Abhilash Raj 08 Mar '15

08 Mar '15

Hi Prakash, On Saturday 07 March 2015 07:20 PM, Prakash kumar wrote: > I want to implement plugin with features:- > > 1. Using regex for filtering texts ( personal information eg: phone > number, address ). > 2. Checking type of files that can be attached to the mails. For eg: > .exe not allowed. > 3. If there are multiple attachments in the email > > If (multiple attachments) then >> for each attachment >> if(not appropriate) then >> discard >> notify sender that this part is removed from email body and why >> else >> continue > > > Is it big enough for a gsoc proposal? Are these one single plugin or 3 different plugins? How are you going to check the type of files? Executables do not have a *nix world. It could be big enough for a GSoC project, but it would depend on how you plan to implement it. Depending on implementation it can be done in a week or a whole summer ;-) > What else I can add to it? If you haven't already fixed a bug, I suggest you to get started on it asap. You will find the links to bug trackers on Ideas Page. > > > > On Thu, Mar 5, 2015 at 7:16 PM, Stephen J. Turnbull <stephen(a)xemacs.org> > wrote: > >> Please keep traffic on-list, unless it's security-related or truly >> personal. Replying to list. >> >> Prakash kumar writes: >> >> > > What does a "plugin" give that improves on this process? >> > >> > The only problem that I find is, we need to implement a framework for >> > loading, unloading and communication of plugins. >> >> We have such a framework already, and since the only person who will >> be using it is the site admin (see below), I really don't see why the >> command-line-based download-copy-configure process is excessively >> burdensome. >> >> > Loading and unloading plugins are better than going to code and making >> > changes for every requirement. >> >> Somebody has to write the code. Once it's written, you just copy the >> module into place, and no code needs to be written. >> >> > It will also help new developers to write plugins without actually >> > understanding the complex coding of mailman. >> >> I can't imagine an interesting plugin that requires no understanding >> of Mailman internals. Theming web interfaces, sure, but that's not >> part of Mailman core anymore, that would for Postorius or HyperKitty >> or some other application. What kind of thing are you thinking about? >> Is it really appropriate for a mailing list (vs a blog or web forum)? >> >> > I think it will be better if we can give the users >> >> What users? See next comments. >> >> > ability to load and unload plugins without touching the source code >> >> We already have that, for values of "user" == site admin. With a >> prewritten plugin (handler or rule), the only Python you need to know >> is the syntax of Python lists and strings, and maybe not even that. >> >> > means from the web ui like in wordpress. >> >> I think this is very unlikely to happen. No sane site admin would >> enable such a feature in Mailman as currently implemented, because >> once you have a Python module, you have access to pretty much >> everything inside of Mailman. For example, you could write a plugin >> that looks for private or no-archive lists, saves a month of posts >> to disk, and then spews the lot to Twitter (after checking the >> language and spewing to Weibo instead for Chinese). >> >> The only thing I can think of would be nice to have, maybe the site >> admin could install the handlers/rules manually as now, but add a >> feature to Postorius to allow manipulation of the pipeline from the >> web UI so that handlers or rules could be enabled selectively for >> individual lists. Again I'm not sure that a sane site admin would >> allow this from the web UI, because manipulating the pipeline or rules >> can have a large impact on performance and correctness of behavior. >> If a poorly ordered pipeline caused a list to go rogue, that could >> affect the reputation of the whole site. >> >> >> > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers(a)python.org > https://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/raj.abhilash1%40… > > Security Policy: http://wiki.list.org/x/QIA9 > -- thanks, Abhilash

2 1

Approach for full anonymization of a mailing list
by Rashi Karanpuria 08 Mar '15

08 Mar '15

1) We create a database table with the schema | orig_id | fake_id | thread_name | where whenever the user posts a first message to some thread a new fake random id is generated and stored in this table. For a specific user fake_id remains same for specific thread. 2) We need to remove the old implementation of anonymous list from src/mailman/handlers/cleanse.py instead we create a new handler that implemented before to_outgoing.py in the pipeline. 3) process method in this handler is exclusively for a anonymous list, it won't process other lists. 4) In this new handler we first replace the value of From header with the fake_id corresponding to this thread for the user who sent this message. The fake_id of the format <foo>@<foo1>.com where foo is randomly generated and foo1 is derived from the list name and is same for all fake addresses on this list. Or foo1 can also be unanimously kept as "anonymous" 5) In the header we put the Reply-To option to be the mailing list address. 6) Then we cleanse the header keeping only From, To, Reply-To . N.B.: In this approach personal mails will be bounced by the MTA. Any suggestions or improvements on this would be of great help. Regards Rashi

2 1

Feature Request : Search Feature in Members list
by Bhavesh Goyal 08 Mar '15

08 Mar '15

While going through postorius I noticed we don't yet have a search function in the member's list for a particular list... Search function might come in handy for a maintainer if he wants to search for a particular member based on his email address and wants to check if he's already subscribed or want to unsubscribe him. Also it will allow list maintainer to search for all addresses starting with 'bha' or any other particular keyword...For large member lists of multiple pages, this feature might come in handy and useful for the maintainer... Also, If found to be something good to have it integrated in postorius, i would like to ask for permission to start working on the same. :) -- Regards, Bhavesh Goyal,

1 0

Re: [Mailman-Developers] GSoC
by Abhilash Raj 08 Mar '15

08 Mar '15

Hi Vlad, On Sunday 08 March 2015 01:20 AM, Vlad Shablinsky wrote: > Hi, I'm Vlad Shablinsky. I wrote about some problems with setting up > Mailman some time ago and Abhilash helped me. I'm interested in > contribution to Mailman this summer. I find "GitHub/development tools > integration" and "A Dashboard for Admins/Owners/Moderators" ideas pretty > interesting for me and I would like to work for one of them. > > I have Mailman and Postorius set up and built for development. I'm going to > fix a bug now, but I need some guidance. I've started reading the > documentation, but not everything works fine. > > 1. When I run `tox` in mailman folder I get OSError: > > "test_linked_nonmember_findable_after_posting > (mailman.rest.tests.test_membership.TestNonmembership) ... Traceback (most > recent call last): > File "/Users/vlad/Projects/GSoC/Mailman/mailman/.tox/py34/bin/runner", > line 9, in <module> > load_entry_point('mailman==3.0.0b6', 'console_scripts', 'runner')() > File > "/Users/vlad/Projects/GSoC/Mailman/mailman/src/mailman/bin/runner.py", line > 200, in main > runner = make_runner(*args.runner, once=args.once) > File > "/Users/vlad/Projects/GSoC/Mailman/mailman/src/mailman/bin/runner.py", line > 100, in make_runner > return runner_class(name, slice) > File > "/Users/vlad/Projects/GSoC/Mailman/mailman/src/mailman/runners/rest.py", > line 55, in __init__ > self._server = make_server() > File > "/Users/vlad/Projects/GSoC/Mailman/mailman/src/mailman/rest/wsgiapp.py", > line 163, in make_server > handler_class=AdminWebServiceWSGIRequestHandler) > File > "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/wsgiref/simple_server.py", > line 153, in make_server > server = server_class((host, port), handler_class) > File > "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", > line 429, in __init__ > self.server_bind() > File > "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/wsgiref/simple_server.py", > line 50, in server_bind > HTTPServer.server_bind(self) > File > "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/server.py", > line 133, in server_bind > socketserver.TCPServer.server_bind(self) > File > "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", > line 440, in server_bind > self.socket.bind(self.server_address) > OSError: [Errno 48] Address already in use” Do you have something running on your `9001` port? This error means mailman test server has problems binding to your local 9001 port as evident from the error message. > 2. The second issue is that I get NameError when trying `dump_list` in > `mailman shell` (I follow these instructions > http://gnu-mailman.readthedocs.org/en/latest/src/mailman/model/docs/address… > ) > First you need to import the `dump_list` function to be able to use it. try: from mailman.testing.documentation import dump_list > ">>> dump_list(user.display_name for user in user_manager.users) > Traceback (most recent call last): > File "<console>", line 1, in <module> > NameError: name 'dump_list' is not defined" > > > 3. I've registed a user via Mozilla Persona and I'm able to log in, but I > don't know what to do with it. I just have one field filled with email of > my registred user. Everything is blank. Are the lists added using mailman > shell? When you did `python manage.py syncdb` in postorius_standalone, did you create a superuser? If yes, then use that email address to login and you'd have lots of others options to play with. If not, then simply go to postorius_standlaone directory and execute the command: python manage.py createsuperuser > I havent't started fixing a bug yet, but I suppose I have to use web UI or > the `mailman shell` to see if the bug is fixed or not. I'm supposed to be > able to reproduce the state that causes the bug. I'm not sure that > everything works fine since I have errors described above. Could you guide > me on that? The usual debugging process after fixing a bug varies from person to person. There may be a lot of things that you won't be able to find in Postorius. We generally use a combination of both shell and mailman.client for our purposes. -- thanks, Abhilash

1 0

Re: [Mailman-Developers] [GSoC] Introduction - A Dashboard for Admins/Owners/Moderators
by Abhilash Raj 08 Mar '15

08 Mar '15

Hi Anuj, On Saturday 07 March 2015 10:10 PM, Anuj Bansal wrote: > Hi all, > > I'm Anuj, a computer science undergraduate at Birla Institute of > Technology and Science, Pilani. I'm interested in the project > "Dashboard for Admins/Owners/Moderators". I've already downloaded and > setup my development environment with the help of instructions given > on [1]. > > I know Python and HTML and have been working on it from last 1 year. > I'm also quite familiar with Django framework and have made some > projects using it. Last semester I made a Django project which is > currently been used by my institute [2]. > > I'd like to contribute to Mailman and hence, would like ideas for > patches / code samples which I could write like supporting modules > which may be required for the project or some other bug fixes. Please read through our Ideas Page and you'd find instructions on how to fix bugs for mailman. For detailed replies to this question you can also dig through archives of this list, it has been answered many times. -- thanks, Abhilash

1 0

Some doubts, feature request and bug in postorius
by Ashish Kumar 08 Mar '15

08 Mar '15

Hi, I am new to postorius interface. I was going through it and In some places I was confused and Hence, made below points for discussion. 1. How to register a user through postorius web interface? 2. After login [ Through super user ] , If I go to mailman settings page (My settings) - Profile : Only name is there and every other field like first name, last name, irc name, etc. is empty. No option to edit it. - Add Email Address: There it seems a kind of bug that I am not able to add an email address there. [Please look at screeshot 1 and 2 for details.] 3. To Do List : No option to add To Do. It always says "There are no pending tasks at the present time." [See screenshot 3] Some feature request: 4. In create list and create domain : There are option to create and delete, Shouldn't there be option for Edit list or domain. 5. There is option for mass subscribe. Shouldn't mass un-subscribe be there. Thanks Ashish Kumar irc nick: ashishk

1 0

GSoC
by Vlad Shablinsky 08 Mar '15

08 Mar '15

Hi, I'm Vlad Shablinsky. I wrote about some problems with setting up Mailman some time ago and Abhilash helped me. I'm interested in contribution to Mailman this summer. I find "GitHub/development tools integration" and "A Dashboard for Admins/Owners/Moderators" ideas pretty interesting for me and I would like to work for one of them. I have Mailman and Postorius set up and built for development. I'm going to fix a bug now, but I need some guidance. I've started reading the documentation, but not everything works fine. 1. When I run `tox` in mailman folder I get OSError: "test_linked_nonmember_findable_after_posting (mailman.rest.tests.test_membership.TestNonmembership) ... Traceback (most recent call last): File "/Users/vlad/Projects/GSoC/Mailman/mailman/.tox/py34/bin/runner", line 9, in <module> load_entry_point('mailman==3.0.0b6', 'console_scripts', 'runner')() File "/Users/vlad/Projects/GSoC/Mailman/mailman/src/mailman/bin/runner.py", line 200, in main runner = make_runner(*args.runner, once=args.once) File "/Users/vlad/Projects/GSoC/Mailman/mailman/src/mailman/bin/runner.py", line 100, in make_runner return runner_class(name, slice) File "/Users/vlad/Projects/GSoC/Mailman/mailman/src/mailman/runners/rest.py", line 55, in __init__ self._server = make_server() File "/Users/vlad/Projects/GSoC/Mailman/mailman/src/mailman/rest/wsgiapp.py", line 163, in make_server handler_class=AdminWebServiceWSGIRequestHandler) File "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/wsgiref/simple_server.py", line 153, in make_server server = server_class((host, port), handler_class) File "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", line 429, in __init__ self.server_bind() File "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/wsgiref/simple_server.py", line 50, in server_bind HTTPServer.server_bind(self) File "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/server.py", line 133, in server_bind socketserver.TCPServer.server_bind(self) File "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", line 440, in server_bind self.socket.bind(self.server_address) OSError: [Errno 48] Address already in use” 2. The second issue is that I get NameError when trying `dump_list` in `mailman shell` (I follow these instructions http://gnu-mailman.readthedocs.org/en/latest/src/mailman/model/docs/address… ) ">>> dump_list(user.display_name for user in user_manager.users) Traceback (most recent call last): File "<console>", line 1, in <module> NameError: name 'dump_list' is not defined" 3. I've registed a user via Mozilla Persona and I'm able to log in, but I don't know what to do with it. I just have one field filled with email of my registred user. Everything is blank. Are the lists added using mailman shell? I havent't started fixing a bug yet, but I suppose I have to use web UI or the `mailman shell` to see if the bug is fixed or not. I'm supposed to be able to reproduce the state that causes the bug. I'm not sure that everything works fine since I have errors described above. Could you guide me on that?

1 0