Hi Mailman Hackers!
I wanted to let you know about some very cool work that Abhilash Raj has been doing, and to request your help to review and test it.
You might know that for a long time, Mailman 3's ORM (Object Relational Mapper) layer was provided by Storm[1]. There were a lot of things to like about Storm, especially because it was such a nice, thin layer over SQL so it was very transparent to use and easy to debug. It had some downsides though, and as the project lost momentum, those downsides became more apparent.
These included lack of any Python 3 support, and bugs in the PostgreSQL layer. As I eventually want MM3 to be a Python 3 application, and it looked like Storm would never be ported, I have for a while desperately wanted to port MM3 to SQLAlchemy. At one point early in the MM3 fork, it *was* using SA, but I had some problems with that (details are lost, and unimportant) and switched to Storm. Now, I think it's clear that SA is the best ORM available for Python, and it is Python 3 compatible[2].
Abhilash has done a fantastic port of the MM3 trunk to SA. I have taken his branch, fixed a few test failures, and cleaned up a few things. But really, all the credit goes to Abhilash for his great work.
I have not yet merged the SA branch into trunk, for a few reasons. Abhilash is investigating Alembic as the schema migration layer, and I have not had time to test the branch against PostgreSQL. I would also like to invite folks who have deployed MM3 to give the branch some testing, especially to make sure that the new SA-derived schema is compatible with the hand-written schema used by Storm.
Adopting SA will mean a break in upgrade guarantees. We won't provide migrations from 3.0b4's schema to 3.0b5, which is why I want them to be effectively compatible. After 3.0b5 is released, we'll likely use Alembic to manage any future schema changes. While I'd rather not make this break, I'm convinced that the SA community can handle all this much better than our crufty pile of schema migration code.
If you are able, please test this branch:
$ bzr branch lp:~barry/mailman/abhilash
That's essentially the 3.0 trunk with Abhilash's branch merged in, and my test fixes and clean ups on top of that. I'd like to merge this to trunk, definitely for 3.0b5, so let's say in the next week or so.
Let us know how it goes!
Cheers, -Barry
[1] https://storm.canonical.com/
[2] We are very likely going to adopt Falcon as a replacement for restish. The latter suffers the same problems as Storm; it's not Python 3 compatible, and it is effectively unmaintained upstream. Falcon is really great, and with the following changes, it will easily support restish-style object-based traversal. Storm and restish are the last Python 2-only dependencies of MM3 core.