Hi Developers!

In honor of the penultimate Mailman Day of 2014, I wanted to announce a significant transition in the implementation of Mailman 3 and call out the contributions of two of our members.

For a long while now we've used Storm as the ORM layer in MM3. ORM, meaning "object relational mapper" is the chunk of code that translates between the persistent Python objects in a running MM3 system and the underlying SQL database. Storm was a good choice many years ago because it was a nice, simple, thin layer that was easy to understand, use, and customize.

In the intervening years though, Storm has shown its age. It is no longer effectively maintained upstream, and it does not support Python 3. While MM3 is not yet a Python 3 application, it is my strong desire to eventually make it so. The fact that Storm has no Python 3 port, and is unlikely to, would prevent MM3 from being ported[*].

In a survey of the alternative available standalone Python ORMs, SQLAlchemy stands out as popular, well-maintained, long stable, and Python 3 compatible, and thus I am happy to announce that the MM3 trunk has been ported to SQLAlchemy (SA). The full test suite completes against both SQLite and PostgreSQL as back-ends, and I would expect that with a little bit of work (contributions welcome!) any other back-end supported by SA should work too.

In addition, rather than the ball-of-hack I wrote to migrate the database schema from one version to another, we will now be using Alembic to manage schema migrations.

I must acknowledge two really spectacular contributions from our members.

Abhilash Raj began the SA port almost on a dare by me on IRC and with many discussions in various forums, produced the bulk of the port. He asked great questions, answered mine own with patience, showed good diligence, and the depth of knowledge about MM3 that such a port required. I have never met Abhilash in person but the quality of his work is impressive.

Aurélien Bompard also provided invaluable contributions to this branch. Some of you may recognize Aurélien from his work on Hyperkitty, the fabulous new archiver. Aurélien was able to join us last year at the Pycon sprint, and it was really great to be able to hang out with him. He did an incredible amount of testing and refinement on the SA branch, feeding me patches that fixed a few of the outstanding test failures, especially on PostgreSQL.

Without the contributions of both Abhilash and Aurélien, this port would not have happened. Kudos to them, and my personal appreciation for their patience and enthusiastic good spirits in working with their picky, curmudgeonly project leader, who took much too long to review and merge their work. :)

I encourage the adventurous among you to grab the bzr trunk and give it a whirl. Do let us know here or in the bug tracker about any problems you might encounter, especially if you are upgrading from an existing Storm-based instance of MM3.

At least one more release of the core is planned this year.

Enjoy, -Barry

[*] We have one other known Python 2-only dependency blocking our porting efforts. We use the restish library as our REST/HTTP framework, and while there are many things I love about restish, it falls into the same camp as Storm. It's Python 2-only, and effectively abandoned upstream. Fortunately we have a ready alternative in the Falcon framework. I have a branch of MM3 ported to Falcon ready to go as soon as upstream releases its new version, which will have a few changes I submitted that allows us to implement restish-style object-based traversal. This means we can port to Falcon, keep largely the same REST code in place, and be ready for Python 3.