[Mailman-Users] The last release from the GNU Mailman project

Stephen J. Turnbull turnbull.stephen.fw at u.tsukuba.ac.jp
Sat Feb 29 02:26:43 EST 2020


@mailman-users
I'm going to get into a lot of design here, so I'm moving the thread to
mailman-developers at python.org.  Reply-To set; please respect.  Brian
kept in Reply-To as a courtesy, don't know if he's subscribed over there.

@mailman-developers Brian is planning to develop an alternative web UI
for admin and archives.  I don't see this as a disincentive to
Postorius or HyperKitty development (proposed implementation language
is PHP, so we'll still need something we can support that we can
bundle), and alternatives have always been part of the vision.  I for
one plan to cooperate, and hope others feel the same.  Thread starts
here on mailman-users:

https://www.mail-archive.com/mailman-users@python.org/msg72530.html

@Brian
If you aren't subscribed to mailman-developers, and don't want to
subscribe,, I'll try to keep you in the loop.

Brian Carpenter writes:

 > I agree and understand. The forum side is not being considered at
 > this time until we get the admin interface nailed down. Right now I
 > am looking at Discourse as a motivation for developing the forum
 > side.  I also think for mailing lists to survive in the future,
 > integrating both approaches to communications is what modern users
 > are looking for.

The Python developers looked at Discourse, I think there's actually a
fair amount of activity.  "I think" because I don't participate in
their forum server, not even entirely sure it's Discourse (checked, it
is).  I don't miss it at all.  I don't think the Discourse users miss
mailing lists at all.  There seems to be near zero crosstalk, even
less crosstalk between Discourse and the lists than there is between
the lists and the issue tracker.

I don't know what would happen with better integration.  Discourse
integration of email, uh, "is poor" IMHO, which in my opinion is an
indication that integration is somewhere between very hard and
impossible -- the original author of Discourse seems to be a brilliant
designer and programmer, with plenty of sympathy for user needs.  If
he didn't do it well, it's surely not at all easy.  A lot of people
feel as you do that "both" is the right answer, and there certainly
was a lot of demand for "both" in Python when Discourse was set up
there.

 > I just think the current interfaces and the decision to go with
 > Django has hurt Mailman 3 rather than help it.

That's assuming that the likely alternative was a non-Django framework
rather than "ssh to the host and fire up python mailman.client".  It
was "ssh to the host and fire up python mailman.client", though. ;-)

 > I also mirror Jim's question of who is the "we" in this
 > conversation.

In practice, it was an "I": Barry Warsaw started rewriting the core
around a decade ago.  Then when a beta-ready version became imminent a
few years later, a couple more "I"s, IIRC Terri Oda and Florian Fuchs
wrote much of Postorius (which Barry named), and the Fedora folks did
HyperKitty because they wanted a forum-like archiver it for the Fedora
lists.  For the last few years, both Postorius and HyperKitty have
been maintained and developed by the "Mailman project team", but those
are the folks primarily responsible for the original design decisions
AIUI.

That's how this works.  People see a need, they start hacking on it
without fanfare because they're not committed to it, once they have an
idea of how much work a commitment involves and they're OK with that
they commit and announce, which often has a chilling effect on
independent alternatives, and tends to cut out users who don't know
they need pay attention if they want input to something that will be
available years later (if at all).  I don't know what to do about the
users; Barry did talk about Mailman 3 on-list occasionally, mostly in
response to issues raised about Mailman 2.

 > Why wasn't I invited? :-D

As always in open source, everybody in general is invited, (almost)
nobody gets a personal invitation.[1]  It's unfortunate that the way
things work folks like you and Jim don't find it so easy to pop over
to mailman-developers to find out about these things in advance, but
we also don't want to burden mailman-users with nitty-gritty details
that that may never be relevant to them.

 > What prevents Mailman 3 from replacing Mailman 2 completely?

Mailman 2 ain't broke, so I don't advise people who are happy with
their installations to try to fix it.  Not even by installing Mailman
3. ;-)

 > Is it because of the interfaces for Mailman 3 totally left Mailman
 > 2 behind or was it the decision to use Django?

Mailman 3 cannot be a drop-in replacement for Mailman 2 because by
design Mailman 3 core has no comprehensive administrative or user
access, except via shell access to the Mailman server.  Otherwise, the
only user access is subscribe/unsubscribe by mail, and I don't think
there's any administrative access by mail (maybe moderation can be
enabled? but it would be disabled by default because mail is tres
insecure by default).

 > Cannot Mailman 3 be used as a standalone 'traditional' mailing list
 > without the need for something like Hyperkitty?

You really need Postorius for the administrator, and it's extremely
desirable for the users.  HyperKitty is trivial to avoid: just don't
have archives.  Almost as trivial is to use mail-archive.com or similar.

 > Can Pipermail be modified to work with Mailman 3 as perhaps a
 > stopgap for Mailman 2 users to feel more comfortable with migrating
 > to Mailman 3?

Yes, but why?  I don't know any users who prefer Pipermail to
HyperKitty.  It's the site admins who (occasionally) end up tearing
their hair out over configuration.

Maybe you mean Mailman 2's native CGIs for admin and user
configuration vs. Postorius rather than the Pipermail archiver?

 > I host hundreds of Mailman 2 lists and I cannot get my clients
 > interested in Mailman 3. It doesn't have all of the features that
 > Mailman 2 has when it comes to list settings, at least not visible

This is true.  I think the main interest from users and list admins is
that for some lists HyperKitty is an acceptable alternative for people
who strongly prefer forum-in-a-browser (not denying your point about
its UI, just saying that on some lists it's good enough).  It's not
prominent to most users, but having a single user that can have
multiple addresses and multiple subscriptions is an advantage (this is
supported in core, not just in Postorius).  And for site admins,
having proper support for virtual domains is big (though you may not
see it that way if you're happy enough with cPanel's workaround).

 > and Hyperkitty is just not impressive to look at when it comes to
 > providing a community feel.

Can you be more specific?  User feedback here has mostly been that
HyperKitty is more Bright! Shiny! and has better search than Pipermail
(which has none, natively), and I have no intuition for what you mean
by "community feel".  Again, perhaps you mean "Postorius" rather than
"HyperKitty" for some of this?

BTW, any confusion between the two is natural, IMO.  Most sites (and I
believe the default) are configured to assume that visiting the
archives is frequent, subscription and user management rare.
Therefore the default for the "lists" subdomain is HyperKitty, with
some other URL to access Postorius.  And of course the user thinks of
the whole thing just as "Mailman". :-(  It's not a seamless UX, but
there's usually no explicit divide, either.

 > I want to research to see if it is possible to provide a browser
 > base interface to convert/import a Mailman 2 list into a Mailman 3
 > list without the need of using a command line. Again I am just
 > focusing on the list (admin) settings to be imported at this point
 > not archives into a forum setting.

This can happen -- there's a pretty good script for this already --
but I think at present about 5% of lists require some admin
intervention besides running the script.  I'm not sure what it will
take to get enough of that last 5%.  Mark and Abhilash would have a
better handle on this, I haven't been keeping up with Mailman 3 user
support much lately.

 > > This makes no sense to me.  If your problem with Django is that it's
 > > written in Python, you've got the REST interface.  That's as far as
 > > our responsibility goes.  See "REST is the approach" below.
 > 
 > I assume Django was primarily chosen was because it was written in 
 > Python.

Of course, that was a requirement.  *We* have to maintain the
bundlable web interfaces.  There were no external volunteers to write
an administrative interface.  The Fedora people who wrote HyperKitty
were ultimately funded by Red Hat, a company which has been all in on
Python for more than two decades.  I don't think they were *required*
to use Python, but I'm not at all surprised they had a strong bias.
Also, the mailman.client Python bindings for the REST API were already
available, and provide that API to Python applications.  No such
bindings were available for other languages or platforms.  (I think
there are now node.js bindings for the REST API, but that was a couple
years ago and I don't know if they are maintained or even used.)

There were alternative frameworks; I'm pretty sure Flask (very
minimal) and Pyramid (much more modular than Django), as well as the
venerable Zope of course, were available and all meet the "Python"
desideratum.  The fact is that at that time Django was considered
Python's rival to running on Rails.  It was the current hotness
(AFAIK, it still is in the Python world), and it had an ORM and
plugins for all the stuff we thought we would need for a robust
administrative application (such as social auth).

 > My main problem with Django is you have to handle a 3rd interface
 > with Mailman 3. So we have three: Postorius, Hyperkitty, and Django.

I don't understand the Django part.  After installation I've never
touched django-admin, I've never used the Django web administration
interface, and I recall only a few cases where I needed to mess with
Django's config files at installation.

 > In other words, [I'm aiming at] One Admin Interface To Rule Them All.

If it's not in Python, we won't be able to support its internals, and
that probably means we won't be able to bundle it.  So it won't be the
OAITRTA.  It might be the VBAITRTA, but people will only get it in a
hosted environment such as EMWD or a distro uber-package that defaults
to it.

BTW, even if you eliminate Django, you may still have the problem of
administration for a database backend, depending on how barebones the
VBITRTA is.  Mailman core has no provision for adding additional
profile information to its database, while both Postorius and
HyperKitty do keep such information in separate tables (of course in
most cases the same RDBMS backend is used for all three).  The
archiver will almost certainly want to keep additional information
(thread and author indicies, maybe a keyword and/or full-text search
index) in a backend, although I guess you could stick with Pipermail
or a similar architecture and use static index files for thread,
author, and date views.

 > I am still, btw, saying that about Django because I am having a
 > very difficult time wrapping my head around its logic.

Nothing wrong with that!  Django is quite opinionated, as well as
being remarkably powerful.  But if we'd been a Ruby shop, we would
have written it in Rails, I'm pretty sure, and you'd need to deal with
that.  That's how these things work.

 > I just seems to me there is a lot of confusion with the use of
 > Mailman 3.

There's a lot of confusion with Mailman 2, as well, as a review of the
last year of malman-users would show.  I guess you just don't notice
because you're a hosting service with a long history of Mailman 2 use.
It's email, OF COURSE there's a lot of confusion. :-(

 > The problem is the discrepancies between the versions. Once bounce 
 > processing shows up in Postorius that discrepancy will spread even 
 > further.

Sure.  Despite our complaints about lack of resources, there is a lot
of progress being made.  There's also a ton of change in the email
and web app world in the last five years: DMARC, ARC, social auth,
2-factor auth, SSL and TLS protocol version deprecations.  We have to
keep up with some of them, and we need to prioritize.

 > The confusing state of installation documentation is also a serious
 > problem. I ended up writing my own that I can get consistent
 > results from with installing Mailman 3 on new servers.

Yeah, we tried participating in Google Season of Docs.  It produced
some new docs I'll be integrating soon, but it wasn't what I'd hoped.
Docs that anticipate user problems are just really hard.  Much easier
to improve them in retrospect.  (I'll buy you a drink if out of the
first ten new Mailman 3 site admins you hand your docs to, fewer than
five have problems installing Mailman 3 using them.  I'm sure they're
very good docs, but I doubt that they handle more than one MTA, for
example, or more than one httpd.)

 > So one of my goals in coming up with a new approach is to make the
 > full setup of a Mailman 3 server far easier to do AND document.

An excellent goal, and I'm confident you'll succeed.  It may take
longer than you expect, that's all.

 > That is fine but what I did not see is the use of U.I. designers.

There was one who was starting work on HyperKitty (she said, on the
Fedora lists many moons back).  Don't know what happened to that.
Probably $DAYJOB.  She's very good and in high demand.

 > So where was this encouragement [to provide alternative interfaces]?

The REST interface itself.

It's not like we had money to put behind third party projects, and we
already had Postorius, and an outside project working on HyperKitty.
That's the business model behind volunteer open source.  People come
together to work on projects for their own reasons.  They do what they
feel like, and eventually they leave.  They're strong on producing
code.  They even produce code to automate management of the project,
they even produce code to automatically document the project, but
they're normally not so strong on the actual management or
documentation. ;-)

To be honest, I can't say that for-profit dev organizations are all
that much better in my experience (although my experience is skewed to
very long-lived open source projects, so they've been improving their
workflows and their docs, and cultivating "organizational culture",
for a long time).

 > What larger platform did you have in mind to integrate Mailman
 > lists into?

Anything that needed to distribute mail in a user-configurable
manner.  But *we* weren't going to do the integration.

 > I am still surprised to see no one has come up with a new interface
 > for Mailman 3.

As I wrote earlier, we were and are amazed at the prevalence of sites
that just stand up Mailman 2 and Pipermail with no search, especially
if they have a lot of private lists inaccessible to Google and
friends.  So nobody has a strong incentive to develop new interfaces
as long as we provide something that mostly works.  Until somebody,
such as you, does.

 > Mark is awesome and I have a great working relationship with him
 > via these lists throughout the years. I believe I will have that
 > with him in the future as well.

Awesome indeed (stop blushing, Mark!), and that's why *we* look
forward to him spending more time on Mailman 3. :-)

Steve

Footnotes: 
[1]  I did, once, so I can't say *no*body.



More information about the Mailman-Users mailing list