On Wed, 17 Nov 1999 20:37:58 -0500 (EST) Barry A Warsaw <bwarsaw@cnri.reston.va.us> wrote:

So do you think the Mailman way is better or worse? I'm curious because I'm trying to decide whether I should port Mailman 1.0's bulk mailer code to the new message pipeline.

MailMan's current code is definitely in the "good enough" category.

In the current 1.2 code base (available via the anonCVS), I os.popen() sendmail[1] passing the entire recipient list on the command line, then I pipe the message text to stdin and let the MTA do the rest. There's one complication; if the length of the recipients list is greater than a certain length (current 3000), I chop it up into multiple popens, but I don't do any sorting of the recipient list.

My view is that appropriate and efficient handling of mail for delivery is the domain of the MTA, and should not be the domain of MUA's (in which camp MailMan sorta falls). As such domain sorting is pleasant, it really only acts to further bolster technically lagging mail servers which are in need of new life anyways.

The advantage I see of this is that sendmail can do it's thing asychronously, without keeping the list object locked the entire time. The disadvantage is that Mailman is only aware of delivery problems if the delivery bounces.

Which of course requires a local MTA, which the current design doesn't.

What actually needs to happen (presuming my current understanding of the source is correct) is that the abstraction of MailMan's internal mail queue needs to be finished. Then, a mail broadcast attempt would only stuff messages (cheaply) into the MailMan queue mechanism, and then return, unlocking the list objects. The queue object of course can be lock free (lockless DB's aren't difficult) and you then merely need to run a could of MailMan queue runners to pipe it to the MTA.

One the messages are stuffed, you can then fork a queue runner, which, if there are already sufficient queue runners immediately dies. If more queue runners are needed however, it proceeds to grab messages and stuff them at the MTA in the normal fashion over SMTP.

Note: If you do this adding VERP support becomes a doddle.

I'm leary though of stepping on too much of the MTA's toes -- a good MTA should just do the right thing.

This is perhaps the best point. Spending time to further bolster fading technologies when better services are freely available hardly seems worth it.

Another alternative, which would be less work and delegates all delivery to the MTA, is to just pump all the recips to the local smtpd via smtplib.py. The advantage here is that again we're MTA independent, but the disadvantage is that Mailman's delivery is synchronous with the smtpd. We'd have to be very sure to unlock the list object during this transaction (but watching out for race conditions, locking again if failure status's are handled directly, etc.) More code, more opportunity for bugs.

No no no no no no. MailMan oeprates asynchronously of the MTA, and because as a list server it generates unusual loads in itself, it cannot be subject to the perfoamcen vagaries of the MTA.

Consider the case I used to commonly run into:

Message arrives and is delivered to MailMan.

MailMan explodes that message into a couple thousand more messages (100K+ subscribers).

MailMan attempts to hand off messages to MTA.

MTA refuses connections as system load is too high.

MailMan bitches.

Meanwhile another message arrives at MailMan to be exploded.

Comments?

Nope!

-- J C Lawrence Internet: claw@kanga.nu ----------(*) Internet: coder@kanga.nu ...Honorary Member of Clan McFud -- Teamer's Avenging Monolith...

Return-Path: <owner-mailman-developers@python.org> Delivered-To: mailman-developers@dinsdale.python.org Received: from python.org (parrot.python.org [132.151.1.90]) by dinsdale.python.org (Postfix) with ESMTP id 568F31CD3E for <mailman-developers@dinsdale.python.org>; Wed, 17 Nov 1999 21:14:51 -0500 (EST) Received: from caesar.elte.hu (caesar.elte.hu [157.181.42.42]) by python.org (8.9.1a/8.9.1) with ESMTP id VAA20080; Wed, 17 Nov 1999 21:14:48 -0500 (EST) Received: from localhost (gorgo@localhost) by caesar.elte.hu (8.9.0/8.9.0) with SMTP id DAA50042; Thu, 18 Nov 1999 03:14:16 +0100 Date: Thu, 18 Nov 1999 03:14:15 +0100 (MET) From: Gergely Madarasz <gorgo@caesar.elte.hu> X-Sender: gorgo@caesar.power.elte.hu To: "Barry A. Warsaw" <bwarsaw@cnri.reston.va.us> Cc: Paul Tomblin <ptomblin@xcski.com>, J C Lawrence <claw@cp.net>, Klaubert Herr da Silveira <klaubert@bcb.gov.br>, mailman-users@python.org, mailman-developers@python.org Subject: Re: [Mailman-Developers] Re: [Mailman-Users] optimizing mail delivery In-Reply-To: <14387.22646.76313.150613@anthem.cnri.reston.va.us> Message-ID: <Pine.A32.3.96.991118030855.28574H-100000@caesar.power.elte.hu> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: mailman-developers-admin@python.org Errors-To: mailman-developers-admin@python.org X-BeenThere: mailman-developers@python.org X-Mailman-Version: 1.1+ Precedence: bulk List-Id: Mailman mailing list developers <mailman-developers.python.org>

On Wed, 17 Nov 1999, Barry A. Warsaw wrote:

So do you think the Mailman way is better or worse? I'm curious because I'm trying to decide whether I should port Mailman 1.0's bulk mailer code to the new message pipeline.

Actually I kinda like the mailman bulk mailer. Before I switched from majordomo, I used an external bulkmail package (called from aliases, what mailman doesn't do on delivery so I can't use it with mailman) to help out sendmail. Changing MTA's on a high traffic site can be a real PITA, I wouldn't like it if I had to change it because mailman can't help it out anymore :( So I think it would be nice if you left it as an option.

-- Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/