On Tue, Apr 24, 2012 at 7:20 AM, Barry Warsaw firstname.lastname@example.org wrote:
Thanks for posting this Pierre-Yves!
On Apr 23, 2012, at 08:17 PM, Pierre-Yves Chibon wrote:
mm-core (handles the lists themselves) --send emails to archivers-->
Note that the core doesn't *have* to send an email to the archiver. From the core's perspective, the
IArchiverinterface has three functions:
- add a message to the archive - get a 'permalink' to the message in the archive - get the url to the "top" of the list's archive
Maybe it would be better to call that the archive's "index", "directory", or "table of contents". The archive may not be hierarchically organized.
The important things are 1) calculating the 'permalink' should not require a round-trip with the archiver; 2) the details of adding a message to the archiver are irrelevant to the core.
Yes, yes, Yes, YES, yesyesyesyesyes! I mean, FTW. ;-)
The questions are then:
- how do we store the emails ?
That's not an appropriate question. The archive backend will decide that, and will provide an IArchive function that can be registered with the core and with front ends.
It would be nice if an IArchive-compatible archive provided a way for new frontends to discover it, but I guess that's kinda bootstrappy -- if we have that, then why don't we just serve the results over that channel?
- how do we expose the API ?
- how to make it such that it becomes easy to extend ? (ie: the stats
module wants to read the db, but probably also to store information on it)
No storing, please. The stats module can keep its own db if it wants to, and should be using on-line algorithms in any case so the expense of hitting the archive should be minimal.
I think we've pretty much come to agreement that the core itself doesn't need a full copy of all the messages after it's sent them, but of course, the "prototype" archiver could be used to keep a local copy of everything in a maildir. That could be shared at the lower level (maildir) or through some kind of API in minikitty.
I don't like the idea of having a "minikitty". As is probably apparent (and I apologize for that, my opinion there is really irrelevant) I am not a fan of turning ML archives into a social network. However, I think Pingou and other HyperKitty worker should just do whatever it is they want to do, and do it right. If you really want a solid base set of functionality and only then extensions, maybe a plugin architecture would be the way to go. Or you can specify and implement that base first, then add the extensions. (But the mockup already sports UI for the extensions!)
But if that's not really what you want to do, Clearsilver provides a perfectly good base set for us, and I'll be happy to maintain the GPL3-ed distro-in-the-Mailman-distro if that's how it needs to be. *You* do what makes *you* happy.
On the other hand, having the archiver-core relying on the same system as the core itself would be nicer from a sysadmin pov.
IMHO, premature optimization. Among other things, there isn't going to be a "the" archiver-core. Mailman should provide "a" archiver-core, and I think it should be based on maildir (which is apparently Barry's intuition, too). Theory and implementation of maildir are simple and robust, and that allows us to concentrate on the archiver interface.
The challenge will be speed
IMHO, Mailman should not take responsibility for speed of any archiver backend distributed with Mailman. It just needs to provide a robust storage, and the two points Barry mentions above.