[Mailman-Developers] any interest in a new built-in web-archive? (i.e. pipermail replacement)

[Barry Warsaw]

On Mar 27, 2012, at 08:53 AM, David Jeske wrote:

>Storm-ORM looks like it allows any primary key so that's good. From a quick
>glance it looks very similar to the Clearsilver-odb-orm in syntax and
>function. My primary beef with django-orm (last time I looked at it) was it
>requiring a unique-id primary key which is pretty poor for multi-row fetch
>performance design.

Here's my take on ORMs.

Early on mm3 went through phases with both SQLObject and SQLAlchemy, which I'd
say are the other two most popular Python ORMs that aren't specifically tied
to a CMS.  It's been a few years now, but both had lots of problems that
manifested as unacceptably slow velocity for making changes to Mailman.

What I like about Storm is that it's so lightweight.  When something goes
wrong (e.g. not being able to store flufl.enum.Enums in the database or UTC
timezone-aware datetimes into SQLite), I've always found it pretty trivial to
dig into Storm, find the problem, maybe fix it, or at least easily work around
it.  The well-written, thin-ish layer of Python code makes up for the lack of
comprehensive documentation.  UTSL.

Storm is also heavily used within Canonical so I know who to beg with or
scream at if I need something :).  Seriously though, it powers Launchpad among
other tech, so it's pretty well battle-tested.

But I'm the first to admit I'm not a database expert, so while I can make
stuff work in Storm, it may not be optimal.  Two examples: I've gotten
donations to make some of the queries more efficient and I'll always welcome
those types of contributions.  Second, I know that some of the constraints I
have expressed in the SQLite schema can't be expressed in a more strict
database such as PostgreSQL.  It would be nice for someone to dig in and fix
those.

The other thing to keep in mind is that within the core, I've tried very hard
to again, use formal interfaces between components so that it could almost not
matter.  This hasn't been field tested so maybe I failed, but the idea is that
you could potentially rip out the model and database layers and provide
alternative implementations of IDatabase and other interfaces, and the rest of
Mailman should Just Work.  I try to be very careful not to break those
abstractions, and I think the Zope Component Architecture is the big win
here.

>Clearsilver templating has a slightly more aggressive enforcement of 'no
>code in template' than django, because it has an explicit intermediate
>dataset called HDF (it doesn't allow access to arbitrary python lists,
>structs, classes, and methods). This allows it to work from any language
>(not just python), and makes the binding from the ORM a bit more explicit.
>However, it also means it's a c-module, and I can see the advantage to
>keeping c-compile dependencies to a minimum.

While I want to keep the Mailman core as pure Python, and so far have seen no
reason to think it can't be, we already have some C dependencies in our
stack.  Some of the libraries that Mailman depends on have C extensions that
are either required or improve performance.  This already means that to deploy
a mm3 server you currently need a C compiler (and the "development" bit of
Python, e.g. Python.h).

Now, eventually you'll probably get this all from your distro, so won't have
to worry about it, but that's the state of affairs atm.

>After I finish some quick cleanup and UI changes to CSLA I want to do, I'll
>take a look at MM2/3 and see if I have better questions once I'm informed.

mm2 and mm3 will differ greatly in the parts you're going to care about.  My
suggestion is to forget about mm2 and concentrate on mm3.

Cheers,
-Barry