At 6:56 PM -0500 2004/01/22, Barry Warsaw wrote:
I had been thinking along the lines of the language for specifying the data source as a db connection and a SQL command.
We were working on a method to send a specially MIME-formatted message to identify the various potential bodyparts, a list of recipients and who gets which bodyparts, then a language for specifying the template which pulls in the appropriate bodyparts for the appropriate recipients (and ways to insert bits of information about the user themselves into the message).
That's certainly tricky, but I think it's got to boil down to privilege or authentication. It would still make me nervous to accept such jobs from other than sites I control.
Yup.I'm just hand-waving here ofcourse, and the rest is left as a simple matter of engineering <wink>.
Yeah. Hand-waving. Right.I should mention that I'm specifically interested in increasing the efficiency between Mailman and its local worker MTAs.
The problem is that this is a very small piece of the overall puzzle, and you're talking about a lot of things that others might also want in other areas. MTA authors are going to be primarily concerned about overall optimization of the MTA in general, and certain employers of certain authors might be concerned about things like WAN efficiency over other things.
I can tell you that with some relatively minor modifications to sendmail, the folks at InfoBeat/Mercury Mail found that they got things to the point where sendmail was no longer the bottleneck -- pulling the data out of the database was the big problem, and much more difficult to solve.
These are allsystems under my control so I should be able to tune them, set up privileges, common data source access, etc. to make things work as smoothly as possible until the message hits the external outgoing interface.
You're talking about picking up a small piece of the puzzle which is likely to have significant components which will be difficult or nearly impossible to solve.
I think there are other areas in which I would be inclined to focus my attentions within Mailman, at least as far as efficiency is concerned. Instead of using pickles, try Berkeley db b-trees, and use that as your "queue" to be processed. The reason here is that b-trees are designed for lightning-fast cursor access, and all you need to do is make sure it's indexed on certain key fields. Let Berkeley db take care of the reliability issues (what happens if there's a crash), efficiently caching information in memory for maximum performance, etc....
I'm not even touching the 3rd rail of putting the MTA /in/ Mailman any more :).
That solution is called L-Soft Listserv, with LSMTP. In fact, you're talking about some of the same sorts of things that they do, only you don't want to own both pieces of code that are implementing the desired standard, which will make solving the problem orders of magnitude more difficult.
-- Brad Knowles, <brad.knowles@skynet.be>
"They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin, Historical Review of Pennsylvania.
GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+ !w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++) tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++)