Mailman 3 January 2015 - Mailman-Developers

Re: [Mailman-Developers] RELEASED: GNU Mailman (core) 3.0b5
by Barry Warsaw 07 Jan '15

07 Jan '15

On Jan 07, 2015, at 11:42 PM, Paul Wise wrote: >What about just a backport of Python 3.4 (and any future mailman3 package)? > >http://backports.debian.org/ For sure, Python 3.4 should be in backports, IMHO. Cheers, -Barry

1 0

Re: [Mailman-Developers] RELEASED: GNU Mailman (core) 3.0b5
by Tanstaafl 07 Jan '15

07 Jan '15

On 1/7/2015 2:13 AM, Paul Wise <pabs3(a)bonedaddy.net> wrote: > On Wed, Jan 7, 2015 at 3:09 PM, Tanstaafl wrote: >> Not for those wanting to avoid systemd. > systemd is optional and easy to avoid in Debian. Until it isn't, and that is the overriding concern, and with very good reason considering the relatively short track record that systemd has.

1 0

Re: [Mailman-Developers] RELEASED: GNU Mailman (core) 3.0b5
by Tanstaafl 07 Jan '15

07 Jan '15

On 1/6/2015 8:36 PM, Paul Wise <pabs3(a)bonedaddy.net> wrote: > On Wed, Jan 7, 2015 at 1:48 AM, Tanstaafl wrote: > >> So, wheezy admins will be left out in the cold. > > I expect Debian jessie will become Debian stable soon enough. Not for those wanting to avoid systemd.

2 1

Some raw thinking on user level authorization and authentication
by Andrew Stuart 06 Jan '15

06 Jan '15

This is as much gathering my thoughts on research. Anyone with questions or comments or ideas welcome. Just raw ideas at this stage. The goal is to build user level REST authentication for Mailman on top of the existing REST API. Mailman user REST authentication/authorization. Use Falcon/Talons wherever appropriate for consistency with Mailman. Authentication - establish the identity of the user Authorization - allow or deny access to specific Mailman resources based on the user identity ################Authentication & authorization process overview: step 1: authentication —- HTTP client provides username and password —- username and password sent to Mailman REST server —- if Mailman REST server authenticates username and password: —- authentication token returned to client step 2: authorization -- examines inbound requests —- if request contains valid authentication information, continue —- examine requested Mailman resource -— send REST request to Mailman REST server to find out if this user is allowed to do this resource request —- if user is allowed access to resource, goto step 3 step 3: action At this stage I have three candidates for what happens next: action 1: —- treat the inbound request as an external proxy authentication subrequest action 2: —- directly proxy the request action 3: -- wsgi drop-through - just let the request pass through to the next level in the wsgi stack ################Possible actions action 1: —- treat the inbound request as an external proxy authentication subrequest In which the inbound HTTP request is first handled by an external proxy server (for example Nginx), which first sends a “subrequest” to our authentication/authorization server. If our authentication/authorization server approves the request, it responses 200 OK to the external proxy server which then directly sends the request to the Mailman REST API ** downside of subrequest based proxying —- depends on external proxy (i.e. Nginx) that suports subrequests based authentication http://nginx.org/en/docs/http/ngx_http_auth_request_module.html —- it would be possible to write the pure Python proxy to user this mechanism, making an external subrequesting authenticating proxy an option rather than requirement ** upside of subrequest based proxying: — once the request is authenticated/authorized, it makes alot of sense for a specialised, high performance proxy to handle the proxying directly with the back end. All proxying edges cases and gotchas are left to the specialised proxy (i.e. Nginx) to handle, and proxy performance is as fast as the external proxy server and the Mailman REST API can talk to each other. action 2: —- directly proxy the request In which the request is then sent on to the Mailman REST API and the response returned to the HTTP client **downside of proxying: —- proxying is a minefield of gotchas and edge cases —- places the authenticating proxy in the path of all requests, certainly slower than raw access to Mailman REST API **upside of proxying: —- can be done as pure Python, no reliance on the behaviour of external proxy servers —- the are probably fewer gotchas when dealing with one know API (i.e. Mailman REST API) versus trying to implement a generalised proxy server action 3: -- wsgi drop-through - just let the request pass through to the next level in the wsgi stack In which the inbound HTTP request comes first through the authorization/authentication wsgi layers then directly to the existing Mailman API via wsgi ** downside of wsgi drop through: —- requires execution of the Mailman REST API as a wsgi application, which is potentially complex and more tightly coupled than desirable and would require implementation into the Mailman code base —- put another way, the auth functions will operate entirely via queries to the Mailman REST API, and such decoupling of auth functions makes for a simpler system

2 1

Re: [Mailman-Developers] MM3 REST API testing [was: Python 3]
by Florian Fuchs 06 Jan '15

06 Jan '15

Am 26.12.2014 um 16:38 schrieb Aurelien Bompard: > And mailman.client must be ported too, since it does import stuff from > mailman for testing purposes. I think Postorious is safe, but that must be > verified (I believe it imports the testing framework from mailman.client > and thus must be ported too). I think, this is worth another thread. So far we haven't found a perfect solution for testing packages that rely heavily on the MM3 REST API. As far as mailman.client, Postorius and HK are concerned, testing without *some* core integration doesn't make too much sense IMO, because any changes in the API would go unnoticed. The first solution for that was to start the core (out of process) with a temporary configuration and on a different port before running the tests and tearing it down afterwards. That way mailman.client/Postorius would be tested with a real MM core underneath it. Compatibility with the API was guaranteed, but it was painfully slow and also a little unstable. It's not fun when every typo is punished with a 10 or 20s penalty! That's why I was extremely happy when Aurélien came up with a much better idea a while ago. His solution made use of the webtest package, which made it possible to test the MM3 API wsgi app without the HTTP overhead or the need to start/stop the main MM process. Much faster, more stable, a real delight compared to the situation we had before. Unfortunately, this now means direct "from mailman" imports which will break with incompatible Python versions. Too bad, we didn't think about this (I, for one, was aware of the fact that Barry plans to do a monolingual port at some point in the future. So I could have thought about that earlier. :-/ meh...). Of course, in order to make our tests work with a py3 MM core, we could go back to the slow and no-fun way of testing that we had before. But maybe there's better way... Any ideas? Cheers Flo

4 11

Re: [Mailman-Developers] X-Message-ID-Hash header (was Re: Python 3)
by Barry Warsaw 06 Jan '15

06 Jan '15

On Jan 04, 2015, at 03:10 PM, Jeff Breidenbach wrote: >So the homebrew hash was dumb idea, and will gracefully >retire in favor of the standard mailman hash algorithm. It >was pretty neat to see some usage, though. And Archived-At >still looks like a winner. Definitely. X-Message-ID-Hash was just a way to elaborate on RFC 5064 $3.2, and because Base 32 was chosen, there isn't any need to worry about needing to escaping non-ascii characters in the Message-ID, nor case/letter/number ambiguity, and still have a reasonably short-ish, human readable URL. Mailman guarantees that any message it forwards will have a Message-ID, although it's usually the case that something upstream will also make such a guarantee. The X-Message-ID-Hash is a pure convenience. We could omit it since we still create and add an Archived-At header with the hash as the last path component, and of course with a well-documented algorithm, it can always be calculated from the Message-ID. It's just handy to include it for other system components that want to combine it easily with RFC 2369's List-Archive header. Cheers, -Barry

1 0

Re: [Mailman-Developers] Python 3
by Barry Warsaw 06 Jan '15

06 Jan '15

On Jan 06, 2015, at 02:55 PM, Aurelien Bompard wrote: >> > What if a site wanted to run them on different VMs for example? >> > >Yeah, but certainly there should be a limit to that. What if a site wanted to >run each mailman runner on a different VM for example? There's already some support for that, in a way. The runners with queues already employ an algorithm to safely split the queue space up into 2^n separate runners without the need for locking. They do of course have to have access to the same directory (e.g. over NFS). Also, now that MM3 is backed by SQL, the use of a client/server db like PostgreSQL eliminates another co-location bottleneck. I wouldn't say this latter is thoroughly fleshed out, and I'm not even sure there's a compelling use case, but it may indeed be possible. >Oh well, I guess there's no other way. I hope it's not going to be a mess to >configure, I'd like to avoid having the admin set HK's base URL in 3 >different places. Agreed. Configuration complexity among all the various components is definitely something to be wary of. Cheers, -Barry

1 0

Introduction and GSoC project requests
by Konstantinos N 06 Jan '15

06 Jan '15

Hello mailman devs and Happy New Year. Since this summer i have been looking at the mailman project as it was the first project that attracted me, being a tool i use at my university to collaborate with my fellow sysadmins. I am generally interested in participating in GSoC at some time but first i would like to start contributing in some way so i can develop better programming skills,learn code researching and bugfixing and generally give back to software that helps me do my work better. The last days i asked around in the irc and gathered resources,links and started learning more about the mailman project.In detail i read about the architecture of mailman 3, differences with mailman 2 and tried reading almost anything i could find helpful in the wiki so i can better understand how mailman really works. I haven't set up my own instance of mailman at the moment because of some hardware problems but i have started reading some code in general, plus looking at the easy bugs listed in the wiki. On to the requests now. I saw two project ideas for GSoC 2014 that really interested me and i would like some guidance around them so i could read more on them. Could someone give me some status on the implementation of them at the moment? Were these ideas for mailman 3, mailman 2 or both? The 2 ideas are 1. Full anonymization 2. No-logging Mode Thank you in advance Konstantinos N

1 1

Re: [Mailman-Developers] Python 3
by Barry Warsaw 06 Jan '15

06 Jan '15

On Jan 06, 2015, at 01:37 PM, Aurelien Bompard wrote: >I must also consider how much work it would be to just port HyperKitty & >KittyStore to Python3. All things considered, it may very well be faster and >more reliable. Of course, I encourage any porting to Python 3 just on principle :) and certainly if I can help, let me know. But you may still want to consider a more distributed communication protocol between core and HK. What if a site wanted to run them on different VMs for example? Cheers, -Barry

3 2

Re: [Mailman-Developers] Python 3
by Barry Warsaw 06 Jan '15

06 Jan '15

On Dec 28, 2014, at 01:51 PM, Aurelien Bompard wrote: >> As I mentioned, I think LMTP *could* work, but REST (inside HK) could >> work too. Aurelien, what do you think? > >I'd go with REST, it seems more flexible and we already have nice libraries >for it. REST will be very nice, and I think with something like Falcon, pretty easy to implement. You probably don't need the object-based (i.e. restish-like) traversal stuff, but if so, I'd be happy to split that out into a separately library that HK could use (and keep it bilingual). Keep it simple, but think through the resource tree and leave open some opportunities for expanding the API later, both for use by MM3 and perhaps other scripting applications. Cheers, -Barry

3 2