[Mailman-Developers] about qrunner and locking

[Barry A. Warsaw]

>>>>> "JCL" == J C Lawrence <claw@kanga.nu> writes:

    JCL> I argue similarly.  To echo you Chuq, a primary goal should
    JCL> be ability to integrate.  That't given I'm increasingly
    JCL> coming to question the use of Python pickles in the first
    JCL> place, let alone use of custom database implementations.  I
    JCL> don't see that the value is there for the increased
    JCL> complexity and isolation of the system.  We already have
    JCL> enough problems given the fact that that the membership base
    JCL> is kept in a pickle that I don't see much reason to go
    JCL> further down that rat hole.  Yes, pickles are nice -- for
    JCL> private data that will never be seen or accessed by an
    JCL> external system.

I completely agree that keeping the list database in marshals (not
pickles, actually) is broken.  The question is what to do about it.

As I see it, we're going to have to adopt some kind of database API
and write against that.  The most general -- and most Pythonic -- one
that I know of is ZODB because all you need to do is derive your class
from a Persistent class and you're good to go (modulo a couple of
special rules).  Being able to write Python the way you're used to
writing it is a big win.  With other approaches and dbi's you can hit
very uncomfortable impedance mismatches.  Or you have to write lots of
strings with embedded SQL, etc.

ZODB has backend storages to interface to many different underlying
databases.  The default FileStorage probably isn't appropriate for
Mailman because it is a versioning storage, and we don't need
versioning (so we don't want to pay the hit in performance and disk
usage).  There are gdbm storages, Berkeley db storages, Oracle
storages, etc.  ZEO is, in fact, implemented as a storage.

Something I believe is possible, is the ability to "mount" storages
into your object tree.  If they work the way I think they work, they'd
be pretty cool.  Here's an example of how I envision using this:

Let's say you run a Mailman site that has intranet mailing lists and
extranet mailing lists.  Let's say further that for external lists,
people can either be clients or people who haven't registered with
your site at all except for their email address.  In this scenario,
you could be getting list rosters from three difference sources
(e.g. internal LDAP server, client list in a relational db,
Mailman-only database).  You'd like Mailman's lists to be able to
encorporate any of these users in its list object.

So, a rough design might be that you've got UserRepositories that can
return User objects.  You can then create Rosters that contain
collections of Users, and list objects would contain a list of Rosters
which would make up the membership addresses.  Now, in the above
example you'd have three different UserRepositories (LDAP, client
rdbms, Mailman objects) which would be mounted into the ZODB object
space.  So when a list is crafting its recipient lists, it doesn't
care where the Users are coming from.

This also comes into play when users want to change their address or
delivery options.  Maybe the options are stored in the backend
database, maybe they're stored only in Mailman's db.  It shouldn't
matter, and Mailman's object system should map those into the same
space transparently.  Same for Rosters perhaps, e.g. maybe Rosters
coming from the intranet database aren't writable through Mailman
because the backend database prohibits it.  That's one way to address
the "we're not going to let employees unsubscribe from this list"
issue.

So the real advantage of using ZODB is that it's very friendly to
Python programs, and should provide the right kind of framework for
plugging in all kinds of backend database sources.

-Barry