On Wed, 24 May 2000 20:47:21 -0700 Chuq Von Rospach <chuqui@plaidworks.com> wrote:
At 6:38 PM -0700 5/24/2000, J C Lawrence wrote:
True. My curiosity however is what MTA's do MX sorting, and more particularly, MX collapsing (eg for two different targets that share an MX's among their lowest level). The potential gains there are likely not huge, but could be (guesstimate) noticable for high volume servers with broad standard deviations in their target lists.
I'll have to check into that some time.
but -- as the experts say, the first $500 buys you 90% of the stereo response, and the rest of the money goes into getting you as close to 100% as you can get.
Umm, true. Looking at it again, and doing a quick check of my user base's MXing, I suspect we're dealing with a less than 1% gain. Bigger fish are available. Methinks my brain was farting.
MX sorting is definitely far up into that 90% range, computationally and time expensive, and lots of other stuff can be done first, with more gain, and less effort.
I don't believe that a list server has any business handling MX sorting unless it is also taking responsibility for being the list MTA. As Mailman isn't, its a moot point.
Maybe one thing we need is a definition of what Mailman is and what it isn't. Some kind of target for the size of lists it wants to reasonably support. If it's 5,000 users, it doesn't matter what you do. If it's 50,000, or 500,000, you definitely have different requirements.
While I really have no say here, were I Barry and Co I'd be comfortable with targetting Mailman as able to handle a mid/high 6digit subscriber base list on mid-range PC-class hardware given suitable system configuration. That wouldn't be the target of course, just the "it must be physically able to work here" metric.
(being able to handle a moderately busy 25,000 user list, say 15-30 messages a day...
Average traffic levels are never the problem. Its the bursts you have to worry about, especially given the enforced latency of a moderated list and there resultant likely grouping of broadcasts. I usually end up moderating/approving messages in groups of 5 making bursts of ~5K messages to the MTA (current largest list has a little under 1K subscribers). It is the burst aspect that's possibly the main reason the MTA delivery process needs to be made asynchronous from the rest of the list server.
It'd be nice to be able to say "5 million subscribers in 2 minutes!", but focus on a solid "do most things for most folks" now, and add the high performance/huge list support in 2.5. But leave the hooks in, so we don't have to rewrite later....)
Were delivery to the MTA seperated from the receipt or CGI process (ie mail is received, the RCPT list attached to it, and the tuple placed on a queue for background processing via forked process or cron job), we wouldn't be having this discussion. Its a fairly invasive change to the current Mailman architecture, but making the whole reciept/broadcast aspect asynchronous offers some really pleasant future avenues.
While its a cheap logic, its easy to note that none of the very high volume commercial email sites out there are based on Sendmail (Critical Path, Hotmail, Onelist, EGroups, etc).
Valid points.
<chortle>
Postfix looks like a *real* win, but until I run it through its paces, I won't use it.
Exactly where I'm at on it. I'm about to roll my desktops over to it, and let it stew there for a couple weeks.
I'm doing 400-500,000 an hour out of my mail system without trying too hard, using sendmail 8.9.3, and peaks approaching 900K.
My traffic on Kanga.Nu (hobby lists: http://www.kanga.nu/lists/listinfo/) is bursty and low enough that I just never get any hours with solidly active spools. I average around 30K - 40K deliveries per hour with the MTA sitting idle for much of that hour. 96% of messages are delivered within 60 seconds of hitting the queue, 98.1% within the hour -- we're basically talking a pretty idle mail system.
So eeking out more performance by swapping MTAs is not a priority)
That's one of the main reasons I've been so lackadaisical about moving to Postfix -- I don't really need to. The only thing driving it is my own interest.
As someone who deals with email for a living...
I should probably note at this point that I'm working for Critical Path on their mail systems.
NB as a contractor.
As long as we're into disclosure, I run a bunch of hobby lists at plaidworks.com...
Just been poking around there and noticed that your archives seem to be inop (dead disk). If you're interested I've been messing about with MHonArc and PHP in my spare time and have almost finished getting a setup that:
-- Allows archived messages to be replied to on the web via the archive page (replies post to the list).
-- Templates (PHPLIB) the entire archive appearance. All MHonArc does is the parsing and data extraction.
-- Supports archive searching by MessageID. I've an MTA hack that inserts a MessageID-based URL into all outgoing Mailman list traffic so the user can just hit the URL and be taken to that message in the archives (searches the MHonArc DB, useful for thread reference etc).
Hopefully I'll get something worth public viewing sometime next week.
-- J C Lawrence Home: claw@kanga.nu ----------(*) Other: coder@kanga.nu --=| A man is as sane as he is dangerous to his environment |=--