exhaustion. Or you could store everything in an mbox file with a file offset index. Or perhaps store everything to an nntp server (Twisted would make a nice platform for this <wink>). ... Also, I really want the next generation archiver to do everything through cgi (or equivalent programmatic interface). The ability to massage the messages on the way out to me outweighs the benefits of vending messages directly from the file system.
Well, since you bring this up.... I've been giving this some thought over the last few weeks, since this latest fit of discussions about archivers cropped up. I've written up some code to address the problem to my satisfaction, along with a quick draft manifesto to explain myself. It's too long to inline here, but I put a copy on the web:
http://home.uchicago.edu/~dgc/sw/mmimap/Meanwhile, to cut to the chase: I decided IMAP is the way to handle this, and I've implemented what I need to provide it for both public and private lists. There are scripts to extract authentication material from Mailman, and an IMAP proxy daemon that performs authentication and sets up an environment to hand off to UW-IMAP.
I've tested on our production server with a restricted set of users. No complaints, and all the testers approve of the approach. Our server needs an upgrade before it's powerful enough to do IMAP for 2000 lists (67,000 subscribers), but it's tentatively the way we plan to go. We probably won't enable HTML archival after the upgrade. We already have a webmail product in place, but if we didn't we could just plug that in on the list server to provide the HTTP access.
I realize that IMAP isn't ideal for all sites or lists, but I think it should work well for our purposes, where lists are mostly institutional, and not so public that they need to be Googled.
I'm hoping to get these materials better integrated and documented soon, maybe once I'm back from LISA. But in case anyone is interested in working with them, I've put them up on the web, linked from the above URL. If this were to be a standard solution rather than a local hack, it would probably need some refactoring for other IMAP daemons, for newer MM authenticators, etc. I'm sure I haven't done the best as can be done, and I'd certainly rather see IMAP access to archives be a standard component of (or interface to) list server software, but it's a pleasing start.
-- -D. dgc@uchicago.edu University of Chicago > NSIT > VDN > ENSS > ENSA > You are here . . . . . . . always line up dots