[Mailman-Developers] Requirements for a new archiver

[Peter C. Norton]

On Wed, Oct 29, 2003 at 09:25:53PM +0100, Brad Knowles wrote:
> > Assuming I had a clean slate to start a database project for a mail
> > store, personally I'd much rather prototype it in something like
> > postgresql where I could add data types to deal with email.  I could
> > then make header types, text types, mime types classes, etc.  Then I
> > could test to see if it was a good idea to implement it.
> 
> 	IMO, that would be an exercise in futility.  We've been down this 
> road a million times before.  We don't need to go down it again to 
> know that the result is not likely to be successful, especially when 
> we have alternatives that are proven to work well -- we store the 
> message meta-data in the database, and then the message bodies in an 
> separate message store akin to INN timecaf/timehash "heaps" (see 
> <http://www.shub-internet.org/brad/papers/dihses/lisa2000/sld090.htm>).

It seems like you're only partially agreeing/disagreeing with me
(optimist/pessamist).  Disagreeing: you're saying that using datatypes
in the database which are appropriate to the kind of data being stored
(mail messages) is an excercise in futility.  But, agreeing: that
storing these in a database in another way is OK.  I don't get why
you'd just want to store these as text when you have databases that
can be made more suitable to the problem.
 
> > I think using a standard sql database for doing mail operations is
> > asking for trouble.  Standard databases don't know how to parse
> > rfc822/2822 headers and that means that you've got to either write a
> > whole lot of stored procedures in a clunky query language (or
> > java!?!?!) and then maintain it, or you've got to do it all in the
> > imap/pop3/whatever server which means a whole lot of yammering traffic
> > between the database and the I/P/W server all the time, which == slow.
> 
> 	You don't ask the database to understand or parse RFC2822 headers 
> or messages.  That's up to your application.  You just store data 
> using the formats known to the database, and the message bodies 
> according to the methods above.

So all the parsing happens in the database client side.  Which is slow.

-Peter

-- 
The 5 year plan:
In five years we'll make up another plan.
Or just re-use this one.