Mailman 3 July 2015 - Mailman-Developers

New name needed for authenticating proxy server
by Andrew Stuart 17 Jul '15

17 Jul '15

"Authenticating Proxy Server" - such a catchy and inspiring name. Barry has suggested and I heartily agree that we should solicit ideas from the Mailman community for a better name for it. Your prize will be the great honour of a heartfelt “thanks!” Anyone at all welcome to submit ideas. Stephen Turnbull has suggested: "For the authentication service, does the cuckoo have any natural enemies? Or we could call it some variation on "Cliff Stoll". :-) Or maybe "Barbarian Gate" (from the cover of Cheswick & Bellovin)." as

2 1

Anyone with JavaScript skills up for a challenge....
by Andrew Stuart 16 Jul '15

16 Jul '15

It would be good also if any daring developers with JavaScript skills were willing to have a go at getting something from the authenticating proxy server to display in a browser. If you decide have a go at it, let me know and we can start by doing a short Google Hangouts session in which I can talk you through the process and concepts. Step one would be to get the authenticating proxy server running on your Mailman server There’s almost no documentation I am sorry to say, but there are installation instructions in the readme, and all of the Mailman REST API documentation remains accurate. https://gitlab.com/astuart/mailmania There is a handy list of API operations available here: http://supercoders.com.au/swagger-ui/dist/index.html Note that the routes are entirely accurate but none of the parameters are. At its simplest, your JavaScript code would need to: 1: send a request to the login route (you will need to have a user configured in Mailman) POST /users/{id}/login 2: the login route will send back a JSON web token. 3: you can then send any API request you like to the server but you must provide the JSON web token in the X-Auth-Token: header It will look something like this: X-Auth-Token: eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkZW50aWZpZXIiOiJ0ZXN0c2VydmVyb3duZXJAbWFpbC5leGFtcGxlLm9yZyIsImV4cCI6MTQzNzE2MTIzOH0.UNo72oWoU4B35Bmmco7PeK7K8eyZcu3OHHBsyhwjhrQ. 4: thats it! as

1 0

A rambling tour of the authenticating proxy server
by Andrew Stuart 16 Jul '15

16 Jul '15

If you have an hour or so to kill you can follow the fairly rambling tour that Barry and I went on of the authenticating proxy server here. Sorry it’s not practised or focused but you should be able to glean some info out of it. http://youtu.be/DWDjtWQNx7c If you have questions - fire away.

1 0

Guided tour of the authenticating proxy server source code Friday
by Andrew Stuart 13 Jul '15

13 Jul '15

Hello all Barry asked if I would mind donating the authentication proxy server to the FSF under a GPL license. I thought this is a good idea and it is now at https://gitlab.com/astuart/mailmania and we are now finalising the copyright transfer. Please note that right now I am cleaning up some things so not much point in trying it out as the tests are broken and some bits and pieces are missing but I am working now to get it finalised and hopefully all will be working within a day or two. I am giving Barry a guided tour of the source code of the authenticating proxy server on Friday 17th (Australian Friday 17th) via Google Hangouts. We’ll try to record it but its a good time to get questions answered so you might get more out of joining in. Anyone else interested in joining us? If so, please say what city/timezone you are in so we can find a common time that suits. thanks as

1 0

Mailman 3 status
by Stephen J. Turnbull 13 Jul '15

13 Jul '15

Hi, Mailman Users! Executive summary: In my opinion, Mailman 3 is ready for use in production, but still requires skills and system accesses over and above those required by Mailman 2. Both in principle and in practice, questions about Mailman 3 should be directed to the Mailman Developers list (preferred) or the #mailman channel on Freenode IRC (where there is an intermittent presence of relevant developers). Note to Mailman Developers: Reply-To set to Mailman Developers. If I've made any misstatements that Mailman Users needs to know about, make sure your post is addressed appropriately to Mailman-Users. What Mailman 3 has ------------------ Mailman 3 has a pretty complete reimplementation of Mailman 2 functionality with a few exceptions. However, the implementation has been radically redesigned, dividing responsibility for Mailman services into - Mailman core: managing user databases and message redistribution - Mailman client: the Python API for interacting with the core, which actually communicates via HTTP using a RESTful protocol - Postorius: the web interface for managing sites (including virtual hosts), lists, and subscriptions - HyperKitty: the list archive manager For the exceptions, please see the release announcement or for details ask on Mailman Developers. Note: we consider the functionality provided by the modules listed above to be quite robust already. We're not ready to say you should migrate your 24x7 security announcement list of 100,000 subscribers to Mailman 3 yet, but Mailman 3 is already serving many discussion lists successfully, and the number is increasing at a satisfactory pace. What Mailman 3 lacks so far --------------------------- - The "topics" feature. - Ports of most Mailman 2 add-ons and commonly-used patches. - Performance measurements. There are theoretical reasons to believe that under certain circumstances a "large" Mailman 3 under "heavy" use *might* suffer bottlenecks, but we just don't know yet. - A Mailman 2 to Mailman 3 migration story. We know what to do, and we can tell you how to do it. It's not hard for simple Mailman 2 installations with a few discussion lists, but it's not automatic. - A "simple" installation story. Because Mailman is now divided into three separate projects ("client" is closely bound to "core") with separate development groups and release cycles, installation has become more complex. It is still recommended that users install recent versions with the latest bugfixes (and for special needs, even more recent versions with feature improvements) from the VCS repository on GitLab. We do have the "Mailman Bundler" script which automates the process, but it is as yet lightly tested, and mostly in use by experienced system administrators rather than "mom and pop" list owners. Some users have experienced a bit of frustration, although so far we've been able to provide workarounds "on the spot", and quickly improve the scripts. Often these improvements are provided by users, which indicates the level of experience of typical testers. As yet there are no packages for Mailman 3 in "stable" or "long term support" releases of OS distros, so that avenue is also closed to folks who don't want to be "guinea pigs" (ie, early stage beta testers for installation and migration). - Hosting service (cPanel etc) integration. - User-tested documentation. Again, most of our beta testers are experienced system admins. Very likely they are reading between the lines for details that people whose administration experience is limited to Mailman lists, probably on a cPanel system, won't get. - A support system on Mailman-Users. I'm sure you all are well-aware of how spoiled you are to get support from Mark Sapiro, with occasional contributions from users and other developers. Mark is still the lead developer on Mailman 2, and has not yet gotten a lot of experience with Mailman 3. We do not have an equivalent person for Mailman 3 yet, although there are several developers any of whom might grow into that role.[1] You still really need to contact the developers directly: - Mailman Developers <mailman-developers(a)python.org> - #mailman channel on Freenode IRC. Note: the "lacks" column will be mostly a barrier to non-system- admins, but of course may be a strong "not quite yet" signal even to experienced admins. What Mailman 3 will have after GSoC ----------------------------------- - An experimental implementation of Dynamic Sublists, which aims at a substantial improvement in functionality over the "topics" feature of Mailman 2. - An experimental implementation of a client based on Node.js, allowing web frameworks based on Node.js to talk directly to Mailman 3. - One or more primitive Mailman 2 to Mailman 3 migration stories. Note: Integration of these features will probably happen on VCS branches in September to October, and will be considered ready for beta use in version 3.1. What Mailman 3 plans for users in version 3.1 --------------------------------------------- - A simple installation story, probably based on the current "Mailman Bundler". - A simple 2to3 migration story. Since we don't have any scripts yet, this will still be considered experimental/direct user suppport needed for Mailman 3.1. "Our experienced staff will be on call." (At least on the Mailman Developers list. I personally will push for attention to Mailman Users as well. Finally, there has been a suggestion of "mailman3-users", also -- watch for details in the release announcement :-). - Improved list-owner-oriented documentation. A final note ------------ The information provided above is accurate to the best of my ability, but it has not been checked by the responsible developers. It is provided in hope of may be of use to those considering installation of Mailman 3. If you're still on the fence after reading this post, please do get more accurate information from the responsible developers on the Mailman Developers list. Footnotes: [1] I really can't speak with authority here; this is my impression though.

3 3

Re: [Mailman-Developers] Mailman 3 status
by Mark Sapiro 13 Jul '15

13 Jul '15

On 7/12/15 11:03 PM, Danil Smirnov wrote: > > Do I correctly understand that Mailman 3 also does not include Usenet > gateway support yet? The mail list to Usenet gateway is in the MM 3 core and should work to gate messages from the mail list to Usenet. The 'gate_news' functionality to gate messages from Usenet to the mail list is not implemented yet. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 1

Is there a way to get list of server_owners from REST API?
by Andrew Stuart 13 Jul '15

13 Jul '15

I can see it is possible to make a user a “server_owner”. Thus I can check a user to see if they are a server_owner. Is there a way to get a list of all server_owners? thanks

2 3

is_server_owner set to true regardless of value posted
by Andrew Stuart 12 Jul '15

12 Jul '15

As far as I can tell, when creating users, if the is_server_owner field is provided in the POST data, then it will always be set to true. Is this the intended behaviour? I was trying to create users and explcitly setting it to False and wondering why they were being created as server_owners anyway. I suppose this is a security issue although I imagine its unlikely any applications are using that field yet. The potential problem I suppose is that in the field there may be installations in which users created via POST have is_server_owner set to true. See the commands below to see the problem in action: # DELETE user to make sure its not present (venv3.4)ubuntu@mail:~/mailmania$ curl -X DELETE --header "authorization: Basic cmVzdGFkbWluOnJlc3RwYXNz" http://localhost:8001/3.0/users/mailmanadmin@example.org # create user via POST with POSTed field value is_server_owner=anyoldvaluewilldo (venv3.4)ubuntu@mail:~/mailmania$ curl -X POST --data "email=mailmanadmin(a)example.org" --data "display_name=displayname" --data "is_server_owner=anyoldvaluewilldo" --header "authorization: Basic cmVzdGFkbWluOnJlc3RwYXNz" http://localhost:8001/3.0/users # GET user - you can now see that is_server_owner is true (venv3.4)ubuntu@mail:~/mailmania$ curl --header "authorization: Basic cmVzdGFkbWluOnJlc3RwYXNz" http://localhost:8001/3.0/users/mailmanadmin@example.org {"display_name": "displayname", "password": "$6$rounds=105489$ToJ.XV3yw0Mvee8r$pbMEE/6e1Xw8PPOunQGX1IL21NmNLrSwl3VFwZNtsoxwCjZ7iWZ.SjDmX7rs9nlM7pglz54GFjB8hmn.rOI.d/", "http_etag": "\"7891a3ea8a2c71a67e738aede9b9d2cfbf438073\"", "user_id": 264084494277271879132250546838180918030, "self_link": "http://localhost:8001/3.0/users/264084494277271879132250546838180918030", "created_on": "2015-07-12T05:08:07.320945", "is_server_owner": true} # DELETE user to make sure its not present (venv3.4)ubuntu@mail:~/mailmania$ curl -X DELETE --header "authorization: Basic cmVzdGFkbWluOnJlc3RwYXNz" http://localhost:8001/3.0/users/mailmanadmin@example.org # create user via POST without any POSTed field value for is_server_owner (venv3.4)ubuntu@mail:~/mailmania$ curl -X POST --data "email=mailmanadmin(a)example.org" --data "display_name=displayname" --header "authorization: Basic cmVzdGFkbWluOnJlc3RwYXNz" http://localhost:8001/3.0/users # GET user - you can now see that is_server_owner is false (venv3.4)ubuntu@mail:~/mailmania$ curl --header "authorization: Basic cmVzdGFkbWluOnJlc3RwYXNz" http://localhost:8001/3.0/users {"total_size": 2, "start": 0, "entries": [{"display_name": "Insecure Testuser", "password": "$6$rounds=107421$p9wIIl6zAEHEe5ZC$JH0MhieMJr8URXGsQ5I6zLa3l4whbRsMhjKOntfDlxOrfstxaRvlMgPvuEYU05ptNnSJdSnB43Elw0zAshFux1", "http_etag": "\"6394e37e623cbfa20445201f091c4aabaaa01340\"", "user_id": 309469782534072420914095048754305489000, "self_link": "http://localhost:8001/3.0/users/309469782534072420914095048754305489000", "created_on": "2015-07-12T04:49:17.499529", "is_server_owner": true}, {"display_name": "displayname", "password": "$6$rounds=98938$pOH6NcMh5RN1mNs0$8wUWsaEXVaqiA6z/B2qTXIYVDMVMunzaMmWCL3JmxL/EnknA4A0PFisJZA0VcDOzY0KqiU2sljlBPXuNrEypl.", "http_etag": "\"3c81af299b5b36b104d9c7066fed63273a72bf48\"", "user_id": 176959263898569148772584820722098620353, "self_link": "http://localhost:8001/3.0/users/176959263898569148772584820722098620353", "created_on": "2015-07-12T05:09:12.043209", "is_server_owner": false}], "http_etag": "\"943acca65a65afd2c1c40b6fc1e010b337e1bad6\”"}

2 1

Update 3.1 Implementation details, changes and request for code review
by Pranjal Yadav 11 Jul '15

11 Jul '15

Hello I recently added another post in my blog as post mid term report. I have added all implementation details and changes made so far, I request you to go through it when ever you find time and kindly update me for any changes required. Blog: prany.github.io

1 0

MM3/HyperKitty and importing Malman2 lists
by Isaac Bennetch 10 Jul '15

10 Jul '15

Hi, Thanks for the good work bringing us Mailman3. I'm doing a new install, which is intended to import data from a different server which had been running Mailman2. I've used the mailman-bundler to install and have a few questions. At this point, I'm still using the SQLite backend but have hopes of using MySQL (the server already has MySQL but not PostgreSQL, and the manual seems to imply that sticking with SQLite may not be ideal). I'm currently attempting to import the old mbox into Hyperkitty[0], which I think I've adapted in to > ./bin/django-admin hyperkitty-import --pythonpath eggs/HyperKitty-1.0.1-py2.7.egg/hyperkitty/ -l testing(a)example.org /home/mailman/old-list.mbox Which doesn't say much (full output below[1]), but didn't fully work. Out of approximately 18300 messages, only six are visible to me through the "Mailing-List Archives" web page, and one of them has a very strange date. It is the second message of the mbox, from 9 Apr 2001, but appears in the web page with approximately the date/time of the import: 9 July 2015 7:27 p.m. Additionally, the remaining five messages are all the most recent -- with the exception of the most recent message not appearing at all. Additionally, if I use the Search box to search for a word that I know appears in (at least) one message, I get no results found. I feel I'm missing something here, any idea what it could be? Furthermore, I haven't found any substantial documentation on importing a subscriber list from Mailman2; is there anything that will allow me to transfer subscribers, their digest preference, and so on without requiring them all to resubscribe manually? Thanks in advance, ~isaac 0 - https://hyperkitty.readthedocs.org/en/latest/development.html 1 - > Importing from mbox file /home/mailman/old-list.mbox to testing(a)example.org > /No 'Message-Id' header in email from None about None > -No 'Message-Id' header in email from None about None > \No 'Message-Id' header in email from None about None > |No 'Message-Id' header in email from None about None > /No 'Message-Id' header in email from None about None > -No 'Message-Id' header in email from None about None > \No 'Message-Id' header in email from None about None > |No 'Message-Id' header in email from None about None > /No 'Message-Id' header in email from None about None > -No 'Message-Id' header in email from None about None > \No 'Message-Id' header in email from None about None > |No 'Message-Id' header in email from None about None > Computing thread structure > Synchronizing properties with Mailman > /usr/local/src/mailman-bundler/eggs/Django-1.7.8-py2.7.egg/django/db/models/fields/__init__.py:1282: RuntimeWarning: DateTimeField MailingList.created_at received a naive datetime (2015-07-04 14:30:22.803087) while time zone support is active. > RuntimeWarning) > > WARNING /usr/local/src/mailman-bundler/eggs/Django-1.7.8-py2.7.egg/django/db/models/fields/__init__.py:1282: RuntimeWarning: DateTimeField MailingList.created_at received a naive datetime (2015-07-04 14:30:22.803087) while time zone support is active. > RuntimeWarning) > > The full-text search index will be updated every minute. Run the 'manage.py runjob update_index' command to update it now.

1 0