[Mailman-Developers] Huge lists

[Chuq Von Rospach]

At 3:41 PM -0700 5/24/2000, J C Lawrence wrote:

>SMTP_MAX_RCPTS needs to be there as some MTAs have internal limits
>on the number of RCPT TOs.  It used to be the old BitNet and big
>iron machines that were most known for this.  Don't know wha't out
>there now, tho I remember getting a couple bounces for that reason a
>couple years back.

there's also an RFC that documents what the supported limits are, and 
it's a good idea to stay below those...

>Yep, its second guessing the MTA, but its a cheap, cost effective,
>minimal impact guess that has nearly NO punitive effect on mailman
>itself.

yes and no. By bunching stuff together, you help the MTA optimize, 
since it's a safe guess that it's going to (at least) sort by domain 
if it does any kind of connection caching at all.

You could make a good argument that the best way to optimize is to 
create one mail batch per unique hostname, up to SMTP-MAX-RCPTS, at 
which point you split it into num_addrs/SMTP-MAX-RCPTS batches for 
that hostname, and then let the MTA sort if out from there.

>  > I guess that we need a per MTA tuning/configuration document.
>
>Aaaargh.  Yes.

Definitely. Since most of the "performance" issues involve the MTA, 
and the MLM only affects it based on how it stuffs things into the 
MTA.

>Without going and re-reading it, about the only thing I can think to
>add to it would be turning off domain checking for localhost RCPTs
>as per our recent comments if that's not there already.

By the way, I suggest that before people *assume* this is an 
improvement that it be tested, because the domain checking has to be 
done somewhere -- and if you do it at the MLM->MTA transfer, in most 
cases it'll still be cached in DNS and save you waiting for the MTA 
delivery phase. So it's not a matter of avoiding the slowdown, but 
where you put it -- in the MLM->MTA layer where the person managing 
the MLM sees it, or in the delivery phase, where it's more or less 
hidden. But it's still there.

>How about Postfix?  Anybody know?

Postfix is "on the list" for later this summer for me...

>  > Have we got that data for the other MTAs? 
>
>Nope.  User written of course, and well, you know.

Sendmail doesn't, sort of. By default, sendmail sorts the queue based 
on an internal "priority" value based on its view of the "urgency" a 
piece of mail needs to be delivered with. But you can switch sendmail 
to use "hostname", where it'll read all of the pending batches, do a 
domain sort and then try to cache stuff together. Sendmail 8.10 also 
has a "by filename" option, where it doesn't open the qf file at all, 
which for diskbound systems can be a big win (I think. I'm still 
testing out 8.10, and won't move it into production on my big machine 
until next week; ask me again after that, when I start tuning it) -- 
but sendmail also does all sorts of connection caching and stuff that 
confuses the matter, which you can tweak if you want.

Right now, I generally recommend sites doing a lot of mail-list 
traffic process their queues sorted by hostname, but also run some 
cron jobs that force queue runs by age, because otherwise, you can 
have stuff that gets stuck in the queue... This may change under 
8.10. I dunno yet.

>  > Of course we could have comparisons of the suitabilities of
>>  different MTAs if people enjoy a decent flamefest.
>
>Of course not.  Everybody knows that Microsoft Exchange is the one
>true MTA and all else are but pale imitations.

don't even JOKE about that. As someone who deals with email for a 
living, the only system that comes *close* to Exchange in the 
braindead category is Lotus notes. And that's not really close. I 
have seen so much braindamage out of Exchange servers I wish I could 
simply reject any mail that ever touched one....

You might as well drive your computers with a squirrel on a wheel.

-- 
Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com)
Apple Mail List Gnome (mailto:chuq@apple.com)

And they sit at the bar and put bread in my jar
and say 'Man, what are you doing here?'"