Hi Justus!
Justus Winter writes:
I'd like to contribute a little to Mailman, and I'd like to better understand how the Mailman project is doing nowadays. I have gotten a bugfix merged in the past, but I now have what I think is a fairly uncontroversial cleanup merge request that has neither been merged nor has it gotten comments.
I think Abhilash has been very busy with day job and some major changes at home (congratulations!), and I certainly have been busy with that. I hope to have more time for Mailman development starting in April, since I will be retiring from $DAYJOB on March 31. Mark has been good about routine requests, especially merging in contributions from translator. He and I have been mostly concentrating on list-based support (most of my work recently), and relatively urgent bug-fixes (Mark and Abhilash). There are a few other core devs but all core currently inactive.
As FOSDEM is around the corner, are any of you going to be there and are up for a chat?
I will not be. Haven't heard anything from the rest of the crew, but we mostly meet at PyCon. This is kinda difficult this year though as my funding for travel from Japan is gone due to preretirement and Abhilash I believe is in India.
Besides cleanups and bugfixes, there are three things I'd like to do:
- Improve Mailman to better scale down to small installations
Not sure what you can really do about that without rearchitecting. The full suite of daemons is something like 13, including 3 WSGI processes, the master daemon for mailman and about 7 or 8 runners.
But I'm pretty sure people have run Mailman 3 on a Raspberry Pi. How constrained an environment are you aiming for?
- Improve latency of messages
What latency are you observing? My last project was getting about 100,000 incoming per day across 20K lists, two incoming runners, 8 outgoing, 1 each for the other Mailman runners. Never saw more than about 5 seconds dwell in the Mailman system, except when the Mailman to outgoing Postfix SMTP connection started glitching. We fixed that by reconfiguring the Mailman host (in Dallas) to use an MX in the same datacenter instead of one in Boston. (!!) And the normal case with a process where I'd do "ls queue/*" evey 5s was completely empty queues. Stuff just didn't stay around long enough for ls to see it.
I see no reason to suppose you can do much better than that, but again, tell me what you're seeing. I'm not experienced in dealing with Mailman at scale, and that host was quite beefy. Still I have a strong feeling that latency is mostly a communication with MTA issue, not in Mailman 3 itself.
- Implement OpenPGP support
What does that mean?
Here are the things I did so far:
- I have Mailman running with runners in threads instead of processes, but that is in a proof-of-concept stage at this point and needs some cleaning up
I guess this is supposed to address the resource consumption (memory footprint?) issue?
After working with Mailman 3 and Postfix, I've become fond of the HUPD (HUPD of Uncontrolled Proliferation of Daemons) model of application design, at least for email. I feel *much* more comfortable messing with individual daemons this way, knowing that I can't affect the others. I'm not going to object to providing the threaded version if people want it, but I would object to wholesale conversion to that model without a lot of production experience based on it.
(I understand that Mailman is a GNU project that wants copyright assignments, and I have done that in the past for other GNU projects, and would be happy to do that for Mailman, but at the same time I feel like putting up *any* barrier to contributing is unfortunate.)
My experience has been that about 2/3 of resistence has been to any paperwork as such, only about 1/3 to assignment vs. some sort of formal license ("contributor agreement", as the PSF calls it).
As far as Mailman is concerned, a lot of the core code has been completely rewritten for Mailman 3. However, I know that in implementing Mailman 2 features not yet in Mailman 3 I've been at least heavily influenced by Mailman 2 code. Not sure that anybody else has been particularly careful about "clean implementations", although Barry has said that the core of Mailman 3 core is completely rewritten from scratch. In any case, the last time licensing was discussed, the founder (John Viega) was not on board with a separation from GNU and a permissive license, and Barry and I at least are pretty sentimental about that. For those reasons, I believe at at least this generation of Mailman core devs is unlikely to move in that direction.
I will take a look at the work you mention, but it will be a couple of weeks at least before I have useful comments.
Steve