Since I've largely finished up the coding contract that was eating up
a lot of my time, I'm thinking that I'd like to do some coding for
fun. And nothing says fun like trying to fix the Mailman archives! ;)
I'm trying to remember all the things people have suggested for the
archives in the past so I can figure out what needs to be done and
what might be nice to have, and see if this is doable in the time I
have in the foreseeable future.
The big things people wanted most, if I recall correctly, included:
- modernized HTML/CSS/Themes (preferably to match a modernized web
interface... is that all set up now?)
- archive links that won't break if the archive is rebuilt
- better address obfuscation (maybe by generating pages through cgi)
- search
- not adding a billion dependencies to Mailman
Here's the list from the wiki's Mailman 2.2 page: http://
wiki.list.org/display/DEV/Mailman+2.2
* Reconsider using a 3rd-party archiver
* Perhaps URLs to messages should be based on message-ids
instead of message numbers so that regenerating archives can't break
links. This must include backward compatible links
* Ditch direct access and vend all archive messages through CGI
so that we can do address obfuscation, and message deletion, etc. on
the fly (with caching of course, but have to worry about web crawlers).
* Add RSS feed
* Allow for admins to remove or edit messages through the web.
* Move archive threads into another list?
* Put archives in the list/mylist directory.
* Add a search option
* Make archives default template look and feel similar to Web UI
(whatever it looks like after the Summer of Code project is done)
* Make archive templatable (at least by changing CSS) so they
can match people's existing site look-and-feel
* MUAs usually make URLs clickable. An new Archive could be used
when posts are distributed, in the footer, so that each message has a
link to the whole thread in the Archive.
* Present all messages in a thread at once, and offer plaintext
download of the whole thread
* Put messages into a database and/or move away from mbox as the
canonical storage format.
So the questions are:
(1) Is anyone working on this already?
(2) What else is on people's wish lists for a pipermail replacement?
Terri
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I've just merged in my 'setuptools' branch to the official Mailman 3
branch. This means that all the autoconf-based building cruft is
gone, dead, and buried. Can you say "Yay"? :)
Instead, Mailman is now a setuptools based project, which is rapidly
becoming the standard for Python applications and libraries. There
are lots and lots of benefits to this change; for developers, the
most immediate benefit is that there's no more configure/make/
makeinstall dance for every little change. It means you can build a
'development' installation of Mailman and edit and test right in
place. This will hugely reduce the overhead for developing the code.
Also gone are the C wrapper programs, so Mailman is now a pure Python
application. With the change to wsgi-based web integration, and
maildir delivery from the MTA, we really don't need them any more.
You'll notice a bunch of other things have disappeared too, like the
3rd party packages we were distributing in the misc/ directory.
Instead, these are downloaded on demand from the Python Cheeseshop
<http://cheeseshop.python.org>. All the packages Mailman depends on
live in the Cheeseshop so we don't need to distribute them any more.
And now that we're a setuptools-based project, when you build Mailman
(see below), these dependent packages will be automatically
downloaded and installed as necessary. You'll need a net connection
for the initial build, but after that, once the packages are
installed, you're good to go.
To prepare your existing branch for the update, start by doing a
'make distclean' followed by a 'bzr revert'. IMPORTANT - don't do
this if you have uncommitted changes! 'bzr stat' should now report
no changes.
Next, cd into your 'misc' directory and remove the following
directories:
- - Elixir-0.3.0
- - SQLAlchemy-0.3.3
- - setuptools-0.6c3
- - zope.interface-3.3.0.1
These are the unpacked dependent packages and you don't need them any
more (in fact, they'll get in your way now). Next, remove any
residual .pyc files laying around, via:
% find . -name \*.pyc -print | xargs rm
Now do your 'bzr pull' to get the latest 3.0 branch changes. If you
have local modifications, you'll need to do a 'bzr merge' and resolve
any conflicts.
To see if everything's cool, pick a 'development' directory. I
usually use a subdirectory called 'staging' in my 3.0 working tree.
This development directory can be anything, but then do this:
% export PYTHONPATH=<my staging dir>
for me "<my staging dir>" would be `pwd`/staging. Make sure this
directory exists.
Next do this:
% python2.5 setup.py develop --install-dir <my staging dir>
After churning for a bit, downloaded some stuff, etc., look in your
staging directory. You'll have a bin directory there with all the
Mailman command line scripts, but all the code will remain in your
working tree.
You can now do this to run the test suite:
% <my staging dir>/bin/testall
You should see 72 tests passing and no failures.
Though you probably won't get very far, the way this is going to work
in deployed installations is that you'll have to run bin/
make_instance to create some directories and do a few other things
that the configure and Makefile.in's used to do. After running that,
you'll have an etc/mailman.cfg file that you can tweak for your
installation. Note that you don't need to do this for bin/testall
because the testing infrastructure creates a temporary instance that
it uses.
Let me now if you have any problems.
- -Barry
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
iQCVAwUBRqJro3EjvBPtnXfVAQL0OgQAktryl9zYZlZrm9dC644EL2hF+FMHs50v
2JTh2w6UhULAdpj+xneKR04pEfCw6HHNult03jo6NoNjYjMmzpUycDHXj7e0RaKE
mbhMkX2v2b6d01OsMrAbAyWBTZH+2rtmjEKANd/1/+LlIdy3KlJe09m+xgNY9VsV
keEIEdVQcYc=
=J3Mq
-----END PGP SIGNATURE-----
> If you are relying on the sender to do the right thing, then
> why not force them to create proper message-ids?
I think Barry's proposal is essentially a numbers game - e.g.
he's hoping for significantly better results using "Date" in
the calculation than not using it.
http://wiki.list.org/display/DEV/Stable+URLs
I'll try to tease out some more useful stats from some large
datasets this weekend. (I can't just run the python scripts as is
because I don't have python 2.5 in the same place as the data,
I don't keep raw message in mbox format, blah blah blah, but
we'll figure it out).
My hypothesis is "Date" doesn't really buy much, but that's
in part because I have a vested interest in that outcome.
We'll see how the data plays out. And I still think RFC2369
headers are needed in the calculation if cross posted
messages are to be handled correctly.
Jeff
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Now that we've successfully navigated the switch to Bazaar, it's time
to lay out plans for future Mailman releases. I've talked to several
people about what to do about Mailman's future and I'd like to take
this opportunity to describe my thoughts and get your feedback.
First some background.
Mailman 2.1 is (shockingly) four and a half years old, having been
initially released on 30-Dec-2002. The last release in the series,
2.1.9 was made almost a year ago. In the meantime, Mark and Tokio
have been doing a great job maintaining the 2.1 branch, with several
important patches in the tree now that will eventually become
2.1.10. The problem of course is that we can't add any new features
to the 2.1 family <wink>, so we should be thinking about a new major
release.
I've been making good progress on the SQAlchemy/Elixir version, which
will finally get rid of pickles and put Mailman on a Real Database
(tm). It's been clear to me for a while that this branch will have a
unified user database. It simply makes no sense to build the
database back-end without once and for all fixing this design
constraint. I've always said that the unified user database will be
in Mailman 3, and thus this branch is indeed called "Mailman 3.0".
I've been slowly building things back up from the ground floor. The
basic data model is in pretty good shape and I'm taking a religious
test-driven approach to making things work again. But the branch
still needs a lot of work, and I have no ETA for Mailman 3.0.
In the meantime, Andrew Kuchling and others have volunteered to work
on modernizing the Mailman web u/i, and Terri recently started a
thread discussing updates to the archiver. I think it makes sense to
bless these efforts, towards the goal of releasing them in Mailman
2.2. I intend to create an official Mailman 2.2 branch in bzr where
these efforts can land as they mature. My hope of course is that
we'll also be able to use much of this new code for Mailman 3.
I'd like to keep the changes for 2.2 focused on the web u/i and
archiver, with a small number of additional features to be
determined. Mailman 2.2 should see no changes to the basic
architecture or 'database'; we'll continue to use pickles by default
for Mailman 2.2. While I won't rule out other new features, I want
to be very picky about those that are accepted for 2.2, and would not
feel bad at all if we rejected or deferred until 3.0 most of those
proposed. Criteria for other 2.2 features must include minimal code
impact with a high degree of reliability and stability.
I plan on updating the wiki pages to reflect this thinking, but I
would like to get feedback from y'all about the plan. It would be
awesome if we could see a release of Mailman 2.2 some time in late
2007 or early 2008.
Comments, question?
- -Barry
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
iQCVAwUBRo/A0nEjvBPtnXfVAQJ+dwP7BXXLM749qO6CXWQKZw41pFN42jYfN6Kg
LjNAQ9IejAT/TISGrSgk8UyZ9kP6ajnOFvKIfJNTFJdytJg8/lvDQSeW1N0u7sR+
Wp0N1e0qA4qfqLYsqRR9W1MQhecdBO/yEJo8KDsOQdGnpfINSKZ40FUvPEbC40U7
C/T83gS+Vxs=
=JJZS
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Jul 9, 2007, at 7:27 PM, Arne Schwabe wrote:
> At our University we developed a customized mini Interface called
> 'simple' Interface. The normal mailman Interface is still there,
> called
> 'expert admin'. A (non working) demo is here:
> https://lists.uni-paderborn.de/listadm/demo.html The code does not use
> the mailman template system nor does it have multi language abilities.
> It even includes code specific to our installation. (We have a
> membership class that maps users to user in ldap and can create
> dynamic
> list with users from ldap + static users)
>
> But maybe something like this should be included in future Mailman
> installation. Either a static simple interface or even a customizable
> simpe interface that is sufficent for 95% of the people (with well
> chosen defaults for your university/organisation)
This is exactly the kind of thing that would be nice to have. I'd
also like for 3.0 to have a 'simple' and 'expert' list admin
interface. BTW, your page makes me wish either 1) I spoke German :)
or 2) Google Translate could handle your URL!
- -Barry
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
iQCVAwUBRqEiRHEjvBPtnXfVAQIfLwQAr25h5wkRKWtvOAXOVfONnFlQEcyG1rYb
tTomQyD5FlSAZO6MeoPEW04QtwWjv9Q3tpxFlf4tRqaPmkdb6pd6WgLy7xkVlSn5
IlTSQdnehB2CNHOKHNMiXHrl45OxxDU9PDPPYyMhaZDNPzaEfT4ad8xay3ktn2cc
2zmW/oLziUo=
=HJOq
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Sorry, I forgot to cross-post this to mailman-users, so I'm reposting.
- -Barry
Now that we've successfully navigated the switch to Bazaar, it's time
to lay out plans for future Mailman releases. I've talked to several
people about what to do about Mailman's future and I'd like to take
this opportunity to describe my thoughts and get your feedback.
First some background.
Mailman 2.1 is (shockingly) four and a half years old, having been
initially released on 30-Dec-2002. The last release in the series,
2.1.9 was made almost a year ago. In the meantime, Mark and Tokio
have been doing a great job maintaining the 2.1 branch, with several
important patches in the tree now that will eventually become
2.1.10. The problem of course is that we can't add any new features
to the 2.1 family <wink>, so we should be thinking about a new major
release.
I've been making good progress on the SQAlchemy/Elixir version, which
will finally get rid of pickles and put Mailman on a Real Database
(tm). It's been clear to me for a while that this branch will have a
unified user database. It simply makes no sense to build the
database back-end without once and for all fixing this design
constraint. I've always said that the unified user database will be
in Mailman 3, and thus this branch is indeed called "Mailman 3.0".
I've been slowly building things back up from the ground floor. The
basic data model is in pretty good shape and I'm taking a religious
test-driven approach to making things work again. But the branch
still needs a lot of work, and I have no ETA for Mailman 3.0.
In the meantime, Andrew Kuchling and others have volunteered to work
on modernizing the Mailman web u/i, and Terri recently started a
thread discussing updates to the archiver. I think it makes sense to
bless these efforts, towards the goal of releasing them in Mailman
2.2. I intend to create an official Mailman 2.2 branch in bzr where
these efforts can land as they mature. My hope of course is that
we'll also be able to use much of this new code for Mailman 3.
I'd like to keep the changes for 2.2 focused on the web u/i and
archiver, with a small number of additional features to be
determined. Mailman 2.2 should see no changes to the basic
architecture or 'database'; we'll continue to use pickles by default
for Mailman 2.2. While I won't rule out other new features, I want
to be very picky about those that are accepted for 2.2, and would not
feel bad at all if we rejected or deferred until 3.0 most of those
proposed. Criteria for other 2.2 features must include minimal code
impact with a high degree of reliability and stability.
I plan on updating the wiki pages to reflect this thinking, but I
would like to get feedback from y'all about the plan. It would be
awesome if we could see a release of Mailman 2.2 some time in late
2007 or early 2008.
Comments, question?
- -Barry
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
iQCVAwUBRo/BJnEjvBPtnXfVAQL7iwP/TfPPvMsTnrrSxQAlvPjQoR27ySqUYh+P
yZCvGxxp9DgNoFQOWl0mo1QzZ9ozXtiFfIHx4CJLybOis+yuiq+BWtih2MJnGBf7
SzD8qsBOu6N4sE8sn4n0tdmXr1fnh4qnrgTobvBX+3toJtHNGQTEVEZCxiWb5fKq
JsUKDVVvOhQ=
=CVNK
-----END PGP SIGNATURE-----