Mailman 3 February 2020 - Mailman-Developers

before next release: disable backscatter in default installation
by Jo Rhett Nov. 7, 2022

Nov. 7, 2022

Hi. There's a fairly simple problem here that needs to be addressed. And it's mostly a documentation/install problem. I'm hoping we can get this resolved before the next release. PROBLEM: Mailman comes out of the box ready to backscatter spam people. Yes, it's easy enough to fix. But because it comes stock this way, and is documented to install this way, most people install it to do this. Those of us who work in abuse departments are tired of hearing "well that's how Mailman works". We also object to having to teach people how to fix their mailman installations because it's not documented in the current manual. This is *exactly* like Sendmail 14 years ago. We didn't accept it then, and Sendmail fixed the problem. RESOLUTION: Mailman default installation should not backscatter in a default configuration. 1. Don't create backscatter aliases for subscribe/unsubscribe/etc by default. Nearly everyone uses web based signup. 2. Discard or hold messages from non-subscribers by default. I would think that it would be perfectly reasonable to have documentation on how to enable the 1980s-style -request / -subscribe etc aliases. However this documentation should have a note that this is against the AUP of nearly every network provider, and enabling it will likely cause them to get listed in various blacklists as a backscatter source. FYI: I know that this goes against the instincts of many old-time mailing list advocates here. But after dealing with a 10k/hour backscatter DoS my tolerance for this problem is understandably limited. Yes, it was a sweet day back in the 1980s. I was running a mailing list server and several UUCP gateways at the time, so I remember them well. But those days are past, and we need to deal with the reality of today. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness

21 168

Re: [Mailman-Users] The last release from the GNU Mailman project
by Stephen J. Turnbull Feb. 29, 2020

Feb. 29, 2020

@mailman-users I'm going to get into a lot of design here, so I'm moving the thread to mailman-developers(a)python.org. Reply-To set; please respect. Brian kept in Reply-To as a courtesy, don't know if he's subscribed over there. @mailman-developers Brian is planning to develop an alternative web UI for admin and archives. I don't see this as a disincentive to Postorius or HyperKitty development (proposed implementation language is PHP, so we'll still need something we can support that we can bundle), and alternatives have always been part of the vision. I for one plan to cooperate, and hope others feel the same. Thread starts here on mailman-users: https://www.mail-archive.com/mailman-users@python.org/msg72530.html @Brian If you aren't subscribed to mailman-developers, and don't want to subscribe,, I'll try to keep you in the loop. Brian Carpenter writes: > I agree and understand. The forum side is not being considered at > this time until we get the admin interface nailed down. Right now I > am looking at Discourse as a motivation for developing the forum > side. I also think for mailing lists to survive in the future, > integrating both approaches to communications is what modern users > are looking for. The Python developers looked at Discourse, I think there's actually a fair amount of activity. "I think" because I don't participate in their forum server, not even entirely sure it's Discourse (checked, it is). I don't miss it at all. I don't think the Discourse users miss mailing lists at all. There seems to be near zero crosstalk, even less crosstalk between Discourse and the lists than there is between the lists and the issue tracker. I don't know what would happen with better integration. Discourse integration of email, uh, "is poor" IMHO, which in my opinion is an indication that integration is somewhere between very hard and impossible -- the original author of Discourse seems to be a brilliant designer and programmer, with plenty of sympathy for user needs. If he didn't do it well, it's surely not at all easy. A lot of people feel as you do that "both" is the right answer, and there certainly was a lot of demand for "both" in Python when Discourse was set up there. > I just think the current interfaces and the decision to go with > Django has hurt Mailman 3 rather than help it. That's assuming that the likely alternative was a non-Django framework rather than "ssh to the host and fire up python mailman.client". It was "ssh to the host and fire up python mailman.client", though. ;-) > I also mirror Jim's question of who is the "we" in this > conversation. In practice, it was an "I": Barry Warsaw started rewriting the core around a decade ago. Then when a beta-ready version became imminent a few years later, a couple more "I"s, IIRC Terri Oda and Florian Fuchs wrote much of Postorius (which Barry named), and the Fedora folks did HyperKitty because they wanted a forum-like archiver it for the Fedora lists. For the last few years, both Postorius and HyperKitty have been maintained and developed by the "Mailman project team", but those are the folks primarily responsible for the original design decisions AIUI. That's how this works. People see a need, they start hacking on it without fanfare because they're not committed to it, once they have an idea of how much work a commitment involves and they're OK with that they commit and announce, which often has a chilling effect on independent alternatives, and tends to cut out users who don't know they need pay attention if they want input to something that will be available years later (if at all). I don't know what to do about the users; Barry did talk about Mailman 3 on-list occasionally, mostly in response to issues raised about Mailman 2. > Why wasn't I invited? :-D As always in open source, everybody in general is invited, (almost) nobody gets a personal invitation.[1] It's unfortunate that the way things work folks like you and Jim don't find it so easy to pop over to mailman-developers to find out about these things in advance, but we also don't want to burden mailman-users with nitty-gritty details that that may never be relevant to them. > What prevents Mailman 3 from replacing Mailman 2 completely? Mailman 2 ain't broke, so I don't advise people who are happy with their installations to try to fix it. Not even by installing Mailman 3. ;-) > Is it because of the interfaces for Mailman 3 totally left Mailman > 2 behind or was it the decision to use Django? Mailman 3 cannot be a drop-in replacement for Mailman 2 because by design Mailman 3 core has no comprehensive administrative or user access, except via shell access to the Mailman server. Otherwise, the only user access is subscribe/unsubscribe by mail, and I don't think there's any administrative access by mail (maybe moderation can be enabled? but it would be disabled by default because mail is tres insecure by default). > Cannot Mailman 3 be used as a standalone 'traditional' mailing list > without the need for something like Hyperkitty? You really need Postorius for the administrator, and it's extremely desirable for the users. HyperKitty is trivial to avoid: just don't have archives. Almost as trivial is to use mail-archive.com or similar. > Can Pipermail be modified to work with Mailman 3 as perhaps a > stopgap for Mailman 2 users to feel more comfortable with migrating > to Mailman 3? Yes, but why? I don't know any users who prefer Pipermail to HyperKitty. It's the site admins who (occasionally) end up tearing their hair out over configuration. Maybe you mean Mailman 2's native CGIs for admin and user configuration vs. Postorius rather than the Pipermail archiver? > I host hundreds of Mailman 2 lists and I cannot get my clients > interested in Mailman 3. It doesn't have all of the features that > Mailman 2 has when it comes to list settings, at least not visible This is true. I think the main interest from users and list admins is that for some lists HyperKitty is an acceptable alternative for people who strongly prefer forum-in-a-browser (not denying your point about its UI, just saying that on some lists it's good enough). It's not prominent to most users, but having a single user that can have multiple addresses and multiple subscriptions is an advantage (this is supported in core, not just in Postorius). And for site admins, having proper support for virtual domains is big (though you may not see it that way if you're happy enough with cPanel's workaround). > and Hyperkitty is just not impressive to look at when it comes to > providing a community feel. Can you be more specific? User feedback here has mostly been that HyperKitty is more Bright! Shiny! and has better search than Pipermail (which has none, natively), and I have no intuition for what you mean by "community feel". Again, perhaps you mean "Postorius" rather than "HyperKitty" for some of this? BTW, any confusion between the two is natural, IMO. Most sites (and I believe the default) are configured to assume that visiting the archives is frequent, subscription and user management rare. Therefore the default for the "lists" subdomain is HyperKitty, with some other URL to access Postorius. And of course the user thinks of the whole thing just as "Mailman". :-( It's not a seamless UX, but there's usually no explicit divide, either. > I want to research to see if it is possible to provide a browser > base interface to convert/import a Mailman 2 list into a Mailman 3 > list without the need of using a command line. Again I am just > focusing on the list (admin) settings to be imported at this point > not archives into a forum setting. This can happen -- there's a pretty good script for this already -- but I think at present about 5% of lists require some admin intervention besides running the script. I'm not sure what it will take to get enough of that last 5%. Mark and Abhilash would have a better handle on this, I haven't been keeping up with Mailman 3 user support much lately. > > This makes no sense to me. If your problem with Django is that it's > > written in Python, you've got the REST interface. That's as far as > > our responsibility goes. See "REST is the approach" below. > > I assume Django was primarily chosen was because it was written in > Python. Of course, that was a requirement. *We* have to maintain the bundlable web interfaces. There were no external volunteers to write an administrative interface. The Fedora people who wrote HyperKitty were ultimately funded by Red Hat, a company which has been all in on Python for more than two decades. I don't think they were *required* to use Python, but I'm not at all surprised they had a strong bias. Also, the mailman.client Python bindings for the REST API were already available, and provide that API to Python applications. No such bindings were available for other languages or platforms. (I think there are now node.js bindings for the REST API, but that was a couple years ago and I don't know if they are maintained or even used.) There were alternative frameworks; I'm pretty sure Flask (very minimal) and Pyramid (much more modular than Django), as well as the venerable Zope of course, were available and all meet the "Python" desideratum. The fact is that at that time Django was considered Python's rival to running on Rails. It was the current hotness (AFAIK, it still is in the Python world), and it had an ORM and plugins for all the stuff we thought we would need for a robust administrative application (such as social auth). > My main problem with Django is you have to handle a 3rd interface > with Mailman 3. So we have three: Postorius, Hyperkitty, and Django. I don't understand the Django part. After installation I've never touched django-admin, I've never used the Django web administration interface, and I recall only a few cases where I needed to mess with Django's config files at installation. > In other words, [I'm aiming at] One Admin Interface To Rule Them All. If it's not in Python, we won't be able to support its internals, and that probably means we won't be able to bundle it. So it won't be the OAITRTA. It might be the VBAITRTA, but people will only get it in a hosted environment such as EMWD or a distro uber-package that defaults to it. BTW, even if you eliminate Django, you may still have the problem of administration for a database backend, depending on how barebones the VBITRTA is. Mailman core has no provision for adding additional profile information to its database, while both Postorius and HyperKitty do keep such information in separate tables (of course in most cases the same RDBMS backend is used for all three). The archiver will almost certainly want to keep additional information (thread and author indicies, maybe a keyword and/or full-text search index) in a backend, although I guess you could stick with Pipermail or a similar architecture and use static index files for thread, author, and date views. > I am still, btw, saying that about Django because I am having a > very difficult time wrapping my head around its logic. Nothing wrong with that! Django is quite opinionated, as well as being remarkably powerful. But if we'd been a Ruby shop, we would have written it in Rails, I'm pretty sure, and you'd need to deal with that. That's how these things work. > I just seems to me there is a lot of confusion with the use of > Mailman 3. There's a lot of confusion with Mailman 2, as well, as a review of the last year of malman-users would show. I guess you just don't notice because you're a hosting service with a long history of Mailman 2 use. It's email, OF COURSE there's a lot of confusion. :-( > The problem is the discrepancies between the versions. Once bounce > processing shows up in Postorius that discrepancy will spread even > further. Sure. Despite our complaints about lack of resources, there is a lot of progress being made. There's also a ton of change in the email and web app world in the last five years: DMARC, ARC, social auth, 2-factor auth, SSL and TLS protocol version deprecations. We have to keep up with some of them, and we need to prioritize. > The confusing state of installation documentation is also a serious > problem. I ended up writing my own that I can get consistent > results from with installing Mailman 3 on new servers. Yeah, we tried participating in Google Season of Docs. It produced some new docs I'll be integrating soon, but it wasn't what I'd hoped. Docs that anticipate user problems are just really hard. Much easier to improve them in retrospect. (I'll buy you a drink if out of the first ten new Mailman 3 site admins you hand your docs to, fewer than five have problems installing Mailman 3 using them. I'm sure they're very good docs, but I doubt that they handle more than one MTA, for example, or more than one httpd.) > So one of my goals in coming up with a new approach is to make the > full setup of a Mailman 3 server far easier to do AND document. An excellent goal, and I'm confident you'll succeed. It may take longer than you expect, that's all. > That is fine but what I did not see is the use of U.I. designers. There was one who was starting work on HyperKitty (she said, on the Fedora lists many moons back). Don't know what happened to that. Probably $DAYJOB. She's very good and in high demand. > So where was this encouragement [to provide alternative interfaces]? The REST interface itself. It's not like we had money to put behind third party projects, and we already had Postorius, and an outside project working on HyperKitty. That's the business model behind volunteer open source. People come together to work on projects for their own reasons. They do what they feel like, and eventually they leave. They're strong on producing code. They even produce code to automate management of the project, they even produce code to automatically document the project, but they're normally not so strong on the actual management or documentation. ;-) To be honest, I can't say that for-profit dev organizations are all that much better in my experience (although my experience is skewed to very long-lived open source projects, so they've been improving their workflows and their docs, and cultivating "organizational culture", for a long time). > What larger platform did you have in mind to integrate Mailman > lists into? Anything that needed to distribute mail in a user-configurable manner. But *we* weren't going to do the integration. > I am still surprised to see no one has come up with a new interface > for Mailman 3. As I wrote earlier, we were and are amazed at the prevalence of sites that just stand up Mailman 2 and Pipermail with no search, especially if they have a lot of private lists inaccessible to Google and friends. So nobody has a strong incentive to develop new interfaces as long as we provide something that mostly works. Until somebody, such as you, does. > Mark is awesome and I have a great working relationship with him > via these lists throughout the years. I believe I will have that > with him in the future as well. Awesome indeed (stop blushing, Mark!), and that's why *we* look forward to him spending more time on Mailman 3. :-) Steve Footnotes: [1] I did, once, so I can't say *no*body.

2 1

Periodically getting KeyError File in Mailman 3
by Shashikanth Komandoor Feb. 25, 2020

Feb. 25, 2020

Hi Team, I am able to successfully install and use the Mailman 3 product but even after the installation also while clicking on different links, I am getting KeyError related to File. I tried to regenerate it but could not and not sure when it happens, how frequently it happens, on clicking which link it happens. Please help me to get rid of it. -- Thanks & Regards, Shashi Kanth.K 9052671936

2 5

Admin, Accounts and Hyperkitty Pages are not loading Properly on Setting DEBUG = False
by Shashikanth Komandoor Feb. 25, 2020

Feb. 25, 2020

Dear Team, If I set the DEBUG = False in */var/lib/mailman/mailman-suite/mailman-suite-project/settings.py* file, I don't see the pages of the applications Accounts, Admin and Hyperkitty are loaded properly. After browsing google, I came to know that the pages of CSS, js and images are not loaded if DEBUG is set false. And my webserver is RHEL 7.5 in built httpd. Please do the needful. I am actually new to Python and doing the risk of running customized Maliman 3 on Production. -- Thanks & Regards, Shashi Kanth.K 9052671936

2 1

Mailman 2.1.30rc1 released
by Mark Sapiro Feb. 19, 2020

Feb. 19, 2020

I am pleased to announce the release of Mailman 2.1.30rc1. Python 2.6 is the minimum supported, but Python 2.7 is strongly recommended. This is a routine bug fix release with a few new features. See the attached README.txt for details. Mailman 2.1.30 will be the last release of the Mailman 2.1 branch from the GNU Mailman project. It contains some new strings which are untranslated in most of the i18n translations. If you can help update any of the translations, please contribute your changes. Changes can be submitted via a bzr merge proposal on Launchpad or by sending an updated mailman.po file and or templates directly to me. This will be the last chance to get i18n updates into a release. Mailman is free software for managing email mailing lists and e-newsletters. Mailman is used for all the python.org and SourceForge.net mailing lists, as well as at hundreds of other sites. For more information, please see our web site at one of: http://www.list.org https://www.gnu.org/software/mailman http://mailman.sourceforge.net/ Mailman 2.1.30rc1 can be downloaded from https://launchpad.net/mailman/2.1/ https://ftp.gnu.org/gnu/mailman/ https://sourceforge.net/projects/mailman/ -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 2

Regarding issue #553 in mailman-core
by ritwik p Feb. 19, 2020

Feb. 19, 2020

The mentioned issue: https://gitlab.com/mailman/mailman/issues/553 So, the line 65 in `eml_confirm.py` in `/src/mailman/commands/` which raises the error is `assert member is None, member` Basically what this line is doing is checking if the `member` is `None` or not and if it ISN'T, it raises an Assertion Error with member details printed. According to the comments (line 57-60) in the same file, member would be `None` when there's an unsubscription, but here it is NOT None (idk why). 2 pretty simple ways of dealing with this will be: - commenting the line - adding an `except AssertionError` line But these won't solve the root problem, i.e why is the member NOT None. I'm currently looking a bit more into it but, How should I proceed with this?? Also, couple of doubts: - How can a moderator confirm the unsubscription request?? - For the tables `user` and `member` in `mailman.db` file in `mailman/var/data/`, when a person unsubscribes from a list, the entry of that member is deleted from the `member` table but not from `user` table. Is it due to the fact that even if the person is not a member, they can still be user?? Thank You

2 13