[Mailman-Developers] ORM layer transition

[Barry Warsaw]

Hi Developers!

In honor of the penultimate Mailman Day of 2014, I wanted to announce a
significant transition in the implementation of Mailman 3 and call out the
contributions of two of our members.

For a long while now we've used Storm as the ORM layer in MM3.  ORM, meaning
"object relational mapper" is the chunk of code that translates between the
persistent Python objects in a running MM3 system and the underlying SQL
database.  Storm was a good choice many years ago because it was a nice,
simple, thin layer that was easy to understand, use, and customize.

In the intervening years though, Storm has shown its age.  It is no longer
effectively maintained upstream, and it does not support Python 3.  While MM3
is not yet a Python 3 application, it is my strong desire to eventually make
it so.  The fact that Storm has no Python 3 port, and is unlikely to, would
prevent MM3 from being ported[*].

In a survey of the alternative available standalone Python ORMs, SQLAlchemy
stands out as popular, well-maintained, long stable, and Python 3 compatible,
and thus I am happy to announce that the MM3 trunk has been ported to
SQLAlchemy (SA).  The full test suite completes against both SQLite and
PostgreSQL as back-ends, and I would expect that with a little bit of work
(contributions welcome!) any other back-end supported by SA should work too.

In addition, rather than the ball-of-hack I wrote to migrate the database
schema from one version to another, we will now be using Alembic to manage
schema migrations.

I must acknowledge two really spectacular contributions from our members.

Abhilash Raj began the SA port almost on a dare by me on IRC and with many
discussions in various forums, produced the bulk of the port.  He asked great
questions, answered mine own with patience, showed good diligence, and the
depth of knowledge about MM3 that such a port required.  I have never met
Abhilash in person but the quality of his work is impressive.

Aurélien Bompard also provided invaluable contributions to this branch.  Some
of you may recognize Aurélien from his work on Hyperkitty, the fabulous new
archiver.  Aurélien was able to join us last year at the Pycon sprint, and it
was really great to be able to hang out with him.  He did an incredible amount
of testing and refinement on the SA branch, feeding me patches that fixed a
few of the outstanding test failures, especially on PostgreSQL.

Without the contributions of both Abhilash and Aurélien, this port would not
have happened.  Kudos to them, and my personal appreciation for their patience
and enthusiastic good spirits in working with their picky, curmudgeonly
project leader, who took much too long to review and merge their work. :)

I encourage the adventurous among you to grab the bzr trunk and give it a
whirl.  Do let us know here or in the bug tracker about any problems you might
encounter, especially if you are upgrading from an existing Storm-based
instance of MM3.

At least one more release of the core is planned this year.

Enjoy,
-Barry

[*] We have one other known Python 2-only dependency blocking our porting
efforts.  We use the restish library as our REST/HTTP framework, and while
there are many things I love about restish, it falls into the same camp as
Storm.  It's Python 2-only, and effectively abandoned upstream.  Fortunately
we have a ready alternative in the Falcon framework.  I have a branch of MM3
ported to Falcon ready to go as soon as upstream releases its new version,
which will have a few changes I submitted that allows us to implement
restish-style object-based traversal.  This means we can port to Falcon, keep
largely the same REST code in place, and be ready for Python 3.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20141101/7f805115/attachment.sig>