[Mailman-Developers] Some notes on my stuff...
Mon, 23 Jul 2001 19:56:15 -0700
On Monday 23 July 2001 08:41, Bill Bradford wrote:
> On Sun, Jul 22, 2001 at 10:15:15PM -0700, Chuq Von Rospach wrote:
> > My testing showed the major processing delay was the speed of uptake
> > by sendmail, primarily due to DNS delays. And given that you can't
> > turn off DNS lookups without turning off a lot of the anti-spam stuff,
> > you're kinda stuck.
> I went from sendmail to postfix, and saw at least a 10x speed
> improvement (from 12-hours-max from sending an email to a list to
> receiving it, to 12 *minutes*.) Turns out Sendmail was choking on the
> "backlog" queue of deferred/undeliverable email. Postfix basically says
> "okay, I"ll try this later, and i'll deliver the email now that CAN be
>From some of the "announcement only" lists that I manage for my customers,
we found similar things, but managed to do it by firing up different
copies of Sendmail periodically to run the mail queue. We too had similar
problems with Sendmail stalling when it'd hit some domain that didn't
resolve properly or that took forever to connect to, and it slowed the
whole process right down.
After doing some tweaking and twiddling, we've now got Sendmail configured
with several different types of queue runners that we fire up via cron:
- One that runs with really _short_ timeouts, to go deep into the mail
queue and send stuff out as quick as possible, without bothering to wait
for long DNS resolves or slow net connections.
- One that runs with really _long_ timeouts, to try to process those msgs
going to hosts that just happen to be on the end of really slow links on
- And a whole bunch in the middle that we fire up to process mail based on
destination domain. By doing a "mailq" and grabbing/counting the
destination domains, we can figure out where most of the mail is going,
and then use "sendmail -qR" to fire up one (or more) queue runners to
deliver mail out to those domains.
It did take us a bit of twiddling to get the queue runners to fire up and
send things out at full tilt without overloading the machine, but after we
got those wrinkles all worked out, we found that Sendmail worked extremely
well (and _MUCH_ better than it did with an "out of the box" config).
Also found that two Sendmail config options that made a _TON_ of
difference, were "HostStatusDirectory" and "SingleThreadDelivery". The
first one lets you share your host up/down status amongst all of the
running Sendmail procs, while the other keeps you from overloading any one
particular destination server by only allowing a single connection to it
from your end.
And lastly, I think I've got to agree wholeheartedly with Chuq on his
notes about spindle usage and IO bandwidth being one of (if not _the_) big
limiting factors in delivering mail. This same machine I've discussed
above saw a fourfold improvement in mail throughput by pulling the mail
spool off onto an underused spindle.
I know Sendmail is generally chastized as being slow and doggish as far as
SMTP servers go, but from having twiddled with it enough lately, I'm
beginning to wonder if thats just because the default config that it
builds with is sluggish (as I've been able to get ~8-10x throughput from
the same server just from reshuffling and retuning the config and the