"PT" == Paul Tomblin <ptomblin@xcski.com> writes:
PT> Wrong. Sendmail *does* do this for you. If you have a bunch
PT> of people on a list in random order, sendmail will work
PT> through the list in order, but every time it connects to a
PT> domain, it will send one message to *all* the members of the
PT> list at that domain. One of my most popular lists has at
PT> least 30% of its subscribers at panix.com, and another 10% at
PT> best.com. When it was on majordomo, the entire list was
PT> passed off to sendmail in one chunk. Since one of the
PT> panix.com people is first on the list, all the panixians got
PT> their mail first, then a few more people, then all the people
PT> at best.com, etc. Now that I'm using mailman, and it's split
PT> into 5 chunks, I've noticed that one of the chunks is all
PT> panixians, which goes in one message to panix, and one of the
PT> chunks is almost all bestians, which also goes in one message.So do you think the Mailman way is better or worse? I'm curious because I'm trying to decide whether I should port Mailman 1.0's bulk mailer code to the new message pipeline.
In the current 1.2 code base (available via the anonCVS), I os.popen() sendmail[1] passing the entire recipient list on the command line, then I pipe the message text to stdin and let the MTA do the rest. There's one complication; if the length of the recipients list is greater than a certain length (current 3000), I chop it up into multiple popens, but I don't do any sorting of the recipient list.
The advantage I see of this is that sendmail can do it's thing asychronously, without keeping the list object locked the entire time. The disadvantage is that Mailman is only aware of delivery problems if the delivery bounces.
I could see trying to sort the recips on the domain name if the total length is > 3000. I can also see porting the local-smptd delivery scheme of the current bulk mailer, but fixing some of the problems in the 1.0 code base (and there are quite a few). I'm leary though of stepping on too much of the MTA's toes -- a good MTA should just do the right thing.
Another alternative, which would be less work and delegates all delivery to the MTA, is to just pump all the recips to the local smtpd via smtplib.py. The advantage here is that again we're MTA independent, but the disadvantage is that Mailman's delivery is synchronous with the smtpd. We'd have to be very sure to unlock the list object during this transaction (but watching out for race conditions, locking again if failure status's are handled directly, etc.) More code, more opportunity for bugs.
Comments?
-Barry
[1] I actually don't use sendmail, I use postfix, but the code is easily configured to use anything with a sendmail compatible command line interface.
Quoting Barry A. Warsaw (bwarsaw@cnri.reston.va.us):
So do you think the Mailman way is better or worse? I'm curious because I'm trying to decide whether I should port Mailman 1.0's bulk mailer code to the new message pipeline.
Personally, I think you should leave it to the MTA to make the decisions on how to deliver the mail more efficiently. Anybody who wants more parallelism can find an MTA to do that. And that leaves Mailman to concentrate on doing the things it does well, rather than second guessing the MTA.
-- Paul Tomblin, not speaking for anybody. SETI@Home: Finally a *good* way to impress Jodie Foster http://www.setiathome.ssl.berkeley.edu/
On Wed, 17 Nov 1999, Barry A. Warsaw wrote:
Another alternative, which would be less work and delegates all delivery to the MTA, is to just pump all the recips to the local smtpd via smtplib.py. The advantage here is that again we're MTA independent, but the disadvantage is that Mailman's delivery is synchronous with the smtpd. We'd have to be very sure to unlock the list object during this transaction (but watching out for race conditions, locking again if failure status's are handled directly, etc.) More code, more opportunity for bugs.
In my own highly-hacked version of mailman that I use, I do this. Mailman simply drops outgoing messages into the queue directory. A background daemon de-queues the messages and pumps them to the local MTA via SMTP. A cronjob checks the daemon periodically, and starts it if is falls over. All very simple, and has worked very reliably for me for several months.
-The Dragon De MonsyneDragon> In my own highly-hacked version of mailman that I use,
Dragon> I do this. Mailman simply drops outgoing messages into
Dragon> the queue directory. A background daemon de-queues the
Dragon> messages and pumps them to the local MTA via SMTP. A
Dragon> cronjob checks the daemon periodically, and starts it if
Dragon> is falls over. All very simple, and has worked very
Dragon> reliably for me for several months.This would make a nice addition to Mailman, and probably wouldn't be hard to port to the new message pipeline model. How'd you like to look into that?
-Barry
On Fri, 26 Nov 1999, Barry A. Warsaw wrote:
Dragon> In my own highly-hacked version of mailman that I use, Dragon> I do this. Mailman simply drops outgoing messages into Dragon> the queue directory. A background daemon de-queues the Dragon> messages and pumps them to the local MTA via SMTP. A Dragon> cronjob checks the daemon periodically, and starts it if Dragon> is falls over. All very simple, and has worked very Dragon> reliably for me for several months.This would make a nice addition to Mailman, and probably wouldn't be hard to port to the new message pipeline model. How'd you like to look into that?
Sure! I'd be happy to. Probably not today, as I have a rather badcase of the flu, and right at the moment sitting upright for any length of time is rewarded with nausea and a blinding headache :P, but mebbe on sunday I'll be up to it & I'll pull the CVS and see what I can do. It should be fairly simple to port.
PS: good job for doing this cleanup, this is something that's been needed for awhile.
-The Dragon De Monsyne
participants (3)
-
Barry A. Warsaw -
Paul Tomblin -
The Dragon De Monsyne