Mailman 3 July 2007 - Mailman-Developers

Improving the archives
by Terri Oda Oct. 3, 2007

Oct. 3, 2007

Since I've largely finished up the coding contract that was eating up a lot of my time, I'm thinking that I'd like to do some coding for fun. And nothing says fun like trying to fix the Mailman archives! ;) I'm trying to remember all the things people have suggested for the archives in the past so I can figure out what needs to be done and what might be nice to have, and see if this is doable in the time I have in the foreseeable future. The big things people wanted most, if I recall correctly, included: - modernized HTML/CSS/Themes (preferably to match a modernized web interface... is that all set up now?) - archive links that won't break if the archive is rebuilt - better address obfuscation (maybe by generating pages through cgi) - search - not adding a billion dependencies to Mailman Here's the list from the wiki's Mailman 2.2 page: http:// wiki.list.org/display/DEV/Mailman+2.2 * Reconsider using a 3rd-party archiver * Perhaps URLs to messages should be based on message-ids instead of message numbers so that regenerating archives can't break links. This must include backward compatible links * Ditch direct access and vend all archive messages through CGI so that we can do address obfuscation, and message deletion, etc. on the fly (with caching of course, but have to worry about web crawlers). * Add RSS feed * Allow for admins to remove or edit messages through the web. * Move archive threads into another list? * Put archives in the list/mylist directory. * Add a search option * Make archives default template look and feel similar to Web UI (whatever it looks like after the Summer of Code project is done) * Make archive templatable (at least by changing CSS) so they can match people's existing site look-and-feel * MUAs usually make URLs clickable. An new Archive could be used when posts are distributed, in the footer, so that each message has a link to the whole thread in the Archive. * Present all messages in a thread at once, and offer plaintext download of the whole thread * Put messages into a database and/or move away from mbox as the canonical storage format. So the questions are: (1) Is anyone working on this already? (2) What else is on people's wish lists for a pipermail replacement? Terri

16 63

Major updates to Mailman 3.0 branch
by Barry Warsaw Aug. 4, 2007

Aug. 4, 2007

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I've just merged in my 'setuptools' branch to the official Mailman 3 branch. This means that all the autoconf-based building cruft is gone, dead, and buried. Can you say "Yay"? :) Instead, Mailman is now a setuptools based project, which is rapidly becoming the standard for Python applications and libraries. There are lots and lots of benefits to this change; for developers, the most immediate benefit is that there's no more configure/make/ makeinstall dance for every little change. It means you can build a 'development' installation of Mailman and edit and test right in place. This will hugely reduce the overhead for developing the code. Also gone are the C wrapper programs, so Mailman is now a pure Python application. With the change to wsgi-based web integration, and maildir delivery from the MTA, we really don't need them any more. You'll notice a bunch of other things have disappeared too, like the 3rd party packages we were distributing in the misc/ directory. Instead, these are downloaded on demand from the Python Cheeseshop <http://cheeseshop.python.org>. All the packages Mailman depends on live in the Cheeseshop so we don't need to distribute them any more. And now that we're a setuptools-based project, when you build Mailman (see below), these dependent packages will be automatically downloaded and installed as necessary. You'll need a net connection for the initial build, but after that, once the packages are installed, you're good to go. To prepare your existing branch for the update, start by doing a 'make distclean' followed by a 'bzr revert'. IMPORTANT - don't do this if you have uncommitted changes! 'bzr stat' should now report no changes. Next, cd into your 'misc' directory and remove the following directories: - - Elixir-0.3.0 - - SQLAlchemy-0.3.3 - - setuptools-0.6c3 - - zope.interface-3.3.0.1 These are the unpacked dependent packages and you don't need them any more (in fact, they'll get in your way now). Next, remove any residual .pyc files laying around, via: % find . -name \*.pyc -print | xargs rm Now do your 'bzr pull' to get the latest 3.0 branch changes. If you have local modifications, you'll need to do a 'bzr merge' and resolve any conflicts. To see if everything's cool, pick a 'development' directory. I usually use a subdirectory called 'staging' in my 3.0 working tree. This development directory can be anything, but then do this: % export PYTHONPATH=<my staging dir> for me "<my staging dir>" would be `pwd`/staging. Make sure this directory exists. Next do this: % python2.5 setup.py develop --install-dir <my staging dir> After churning for a bit, downloaded some stuff, etc., look in your staging directory. You'll have a bin directory there with all the Mailman command line scripts, but all the code will remain in your working tree. You can now do this to run the test suite: % <my staging dir>/bin/testall You should see 72 tests passing and no failures. Though you probably won't get very far, the way this is going to work in deployed installations is that you'll have to run bin/ make_instance to create some directories and do a few other things that the configure and Makefile.in's used to do. After running that, you'll have an etc/mailman.cfg file that you can tweak for your installation. Note that you don't need to do this for bin/testall because the testing infrastructure creates a temporary instance that it uses. Let me now if you have any problems. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBRqJro3EjvBPtnXfVAQL0OgQAktryl9zYZlZrm9dC644EL2hF+FMHs50v 2JTh2w6UhULAdpj+xneKR04pEfCw6HHNult03jo6NoNjYjMmzpUycDHXj7e0RaKE mbhMkX2v2b6d01OsMrAbAyWBTZH+2rtmjEKANd/1/+LlIdy3KlJe09m+xgNY9VsV keEIEdVQcYc= =J3Mq -----END PGP SIGNATURE-----

2 2

Re: [Mailman-Developers] Improving the archives
by Jeff Breidenbach July 27, 2007

July 27, 2007

> If you are relying on the sender to do the right thing, then > why not force them to create proper message-ids? I think Barry's proposal is essentially a numbers game - e.g. he's hoping for significantly better results using "Date" in the calculation than not using it. http://wiki.list.org/display/DEV/Stable+URLs I'll try to tease out some more useful stats from some large datasets this weekend. (I can't just run the python scripts as is because I don't have python 2.5 in the same place as the data, I don't keep raw message in mbox format, blah blah blah, but we'll figure it out). My hypothesis is "Date" doesn't really buy much, but that's in part because I have a vested interest in that outcome. We'll see how the data plays out. And I still think RFC2369 headers are needed in the calculation if cross posted messages are to be handled correctly. Jeff

1 0

Mailman roadmap
by Barry Warsaw July 23, 2007

July 23, 2007

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Now that we've successfully navigated the switch to Bazaar, it's time to lay out plans for future Mailman releases. I've talked to several people about what to do about Mailman's future and I'd like to take this opportunity to describe my thoughts and get your feedback. First some background. Mailman 2.1 is (shockingly) four and a half years old, having been initially released on 30-Dec-2002. The last release in the series, 2.1.9 was made almost a year ago. In the meantime, Mark and Tokio have been doing a great job maintaining the 2.1 branch, with several important patches in the tree now that will eventually become 2.1.10. The problem of course is that we can't add any new features to the 2.1 family <wink>, so we should be thinking about a new major release. I've been making good progress on the SQAlchemy/Elixir version, which will finally get rid of pickles and put Mailman on a Real Database (tm). It's been clear to me for a while that this branch will have a unified user database. It simply makes no sense to build the database back-end without once and for all fixing this design constraint. I've always said that the unified user database will be in Mailman 3, and thus this branch is indeed called "Mailman 3.0". I've been slowly building things back up from the ground floor. The basic data model is in pretty good shape and I'm taking a religious test-driven approach to making things work again. But the branch still needs a lot of work, and I have no ETA for Mailman 3.0. In the meantime, Andrew Kuchling and others have volunteered to work on modernizing the Mailman web u/i, and Terri recently started a thread discussing updates to the archiver. I think it makes sense to bless these efforts, towards the goal of releasing them in Mailman 2.2. I intend to create an official Mailman 2.2 branch in bzr where these efforts can land as they mature. My hope of course is that we'll also be able to use much of this new code for Mailman 3. I'd like to keep the changes for 2.2 focused on the web u/i and archiver, with a small number of additional features to be determined. Mailman 2.2 should see no changes to the basic architecture or 'database'; we'll continue to use pickles by default for Mailman 2.2. While I won't rule out other new features, I want to be very picky about those that are accepted for 2.2, and would not feel bad at all if we rejected or deferred until 3.0 most of those proposed. Criteria for other 2.2 features must include minimal code impact with a high degree of reliability and stability. I plan on updating the wiki pages to reflect this thinking, but I would like to get feedback from y'all about the plan. It would be awesome if we could see a release of Mailman 2.2 some time in late 2007 or early 2008. Comments, question? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBRo/A0nEjvBPtnXfVAQJ+dwP7BXXLM749qO6CXWQKZw41pFN42jYfN6Kg LjNAQ9IejAT/TISGrSgk8UyZ9kP6ajnOFvKIfJNTFJdytJg8/lvDQSeW1N0u7sR+ Wp0N1e0qA4qfqLYsqRR9W1MQhecdBO/yEJo8KDsOQdGnpfINSKZ40FUvPEbC40U7 C/T83gS+Vxs= =JJZS -----END PGP SIGNATURE-----

5 7

Potential solution to protecting email addresses in archive...
by Dale Newfield July 22, 2007

July 22, 2007

http://mailhide.recaptcha.net/ -Dale D<a href="http://mailhide.recaptcha.net/d?k=01Qtvu7BFKxAunezLXAq0QPA==&c=QjjpEgdd…" onclick="window.open('http://mailhide.recaptcha.net/d?k=01Qtvu7BFKxAunezLXAq0QPA==&c=QjjpEgdd…','','toolbar=0,scrollbars=0,location=0,statusbar=0,menubar=0,resizable=0,width=500,height=300'); return false;" title="Reveal this e-mail address">...</a>@Newfield.org

3 3

Re: [Mailman-Developers] Mailman roadmap
by Barry Warsaw July 20, 2007

July 20, 2007

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 9, 2007, at 7:27 PM, Arne Schwabe wrote: > At our University we developed a customized mini Interface called > 'simple' Interface. The normal mailman Interface is still there, > called > 'expert admin'. A (non working) demo is here: > https://lists.uni-paderborn.de/listadm/demo.html The code does not use > the mailman template system nor does it have multi language abilities. > It even includes code specific to our installation. (We have a > membership class that maps users to user in ldap and can create > dynamic > list with users from ldap + static users) > > But maybe something like this should be included in future Mailman > installation. Either a static simple interface or even a customizable > simpe interface that is sufficent for 95% of the people (with well > chosen defaults for your university/organisation) This is exactly the kind of thing that would be nice to have. I'd also like for 3.0 to have a 'simple' and 'expert' list admin interface. BTW, your page makes me wish either 1) I spoke German :) or 2) Google Translate could handle your URL! - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBRqEiRHEjvBPtnXfVAQIfLwQAr25h5wkRKWtvOAXOVfONnFlQEcyG1rYb tTomQyD5FlSAZO6MeoPEW04QtwWjv9Q3tpxFlf4tRqaPmkdb6pd6WgLy7xkVlSn5 IlTSQdnehB2CNHOKHNMiXHrl45OxxDU9PDPPYyMhaZDNPzaEfT4ad8xay3ktn2cc 2zmW/oLziUo= =HJOq -----END PGP SIGNATURE-----

1 0

Mailman roadmap
by Barry Warsaw July 7, 2007

July 7, 2007

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Sorry, I forgot to cross-post this to mailman-users, so I'm reposting. - -Barry Now that we've successfully navigated the switch to Bazaar, it's time to lay out plans for future Mailman releases. I've talked to several people about what to do about Mailman's future and I'd like to take this opportunity to describe my thoughts and get your feedback. First some background. Mailman 2.1 is (shockingly) four and a half years old, having been initially released on 30-Dec-2002. The last release in the series, 2.1.9 was made almost a year ago. In the meantime, Mark and Tokio have been doing a great job maintaining the 2.1 branch, with several important patches in the tree now that will eventually become 2.1.10. The problem of course is that we can't add any new features to the 2.1 family <wink>, so we should be thinking about a new major release. I've been making good progress on the SQAlchemy/Elixir version, which will finally get rid of pickles and put Mailman on a Real Database (tm). It's been clear to me for a while that this branch will have a unified user database. It simply makes no sense to build the database back-end without once and for all fixing this design constraint. I've always said that the unified user database will be in Mailman 3, and thus this branch is indeed called "Mailman 3.0". I've been slowly building things back up from the ground floor. The basic data model is in pretty good shape and I'm taking a religious test-driven approach to making things work again. But the branch still needs a lot of work, and I have no ETA for Mailman 3.0. In the meantime, Andrew Kuchling and others have volunteered to work on modernizing the Mailman web u/i, and Terri recently started a thread discussing updates to the archiver. I think it makes sense to bless these efforts, towards the goal of releasing them in Mailman 2.2. I intend to create an official Mailman 2.2 branch in bzr where these efforts can land as they mature. My hope of course is that we'll also be able to use much of this new code for Mailman 3. I'd like to keep the changes for 2.2 focused on the web u/i and archiver, with a small number of additional features to be determined. Mailman 2.2 should see no changes to the basic architecture or 'database'; we'll continue to use pickles by default for Mailman 2.2. While I won't rule out other new features, I want to be very picky about those that are accepted for 2.2, and would not feel bad at all if we rejected or deferred until 3.0 most of those proposed. Criteria for other 2.2 features must include minimal code impact with a high degree of reliability and stability. I plan on updating the wiki pages to reflect this thinking, but I would like to get feedback from y'all about the plan. It would be awesome if we could see a release of Mailman 2.2 some time in late 2007 or early 2008. Comments, question? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBRo/BJnEjvBPtnXfVAQL7iwP/TfPPvMsTnrrSxQAlvPjQoR27ySqUYh+P yZCvGxxp9DgNoFQOWl0mo1QzZ9ozXtiFfIHx4CJLybOis+yuiq+BWtih2MJnGBf7 SzD8qsBOu6N4sE8sn4n0tdmXr1fnh4qnrgTobvBX+3toJtHNGQTEVEZCxiWb5fKq JsUKDVVvOhQ= =CVNK -----END PGP SIGNATURE-----

1 0