Without comment on whether this whole approach is correct, I would suggest that you do not want to modify the deliver function. I'm assuming you have not enabled threaded delivery, i.e. set the mm_cfg.MAX_DELIVERY_THREADS to a non-zero value. If you have, you might like to reconsider that decision as it will mean the threaded_deliver function may be being used, assuming your Python is installed to support threads. If it is, that may be part of the problem. I'll also assume you haven't changed the SMTP_MAX_RCPTS to an unreasonable value in mm_cfg.py Looking at the problem you pose; Mailman divides the total recipient list for a mail in to a sequence of chunks in the process function of SMTPDirect.py. By default I think SMTP_MAX_RCPTS=500. It then iterates over that chunk sequence in the process function of SMTPDirect.py, calling the deliver function, which opens a new SMTP connection to pass the message to the SMTP server for each chunk's-worth of recipients. You would be better off putting your delay into the loop over the chunks in the process function, to give some breathing space between connections being opened, rather than delaying the send to the SMTP server once the connection has been opened, which is what your proposal does. Doing the delay in the process function should also have the advantage of less impact on operation when the number of recipients for a message is more modest, if you make the delay conditional on there being a large number of recipients. You could also play some games so that the amount of throttling by Mailman is conditional on the total number of recipients using some suitable algorithm to calculate the delay. Just for fun I've included the content of a samll patch file for SMTPDirect.py below which: 1. adds a function for computing the inter-chunk deliver delay. Change the algorithm to suit your prejudices. 2. adds a conditional sleep between calls to deliver for each chunk of recipients. Somebody will probably say this is all of load of codswallop and they may well be right. But I hope you solve your problem and would be interested to hear how you do so. cut here------------------------------------------------------ --- SMTPDirect.py.orig Fri Sep 6 10:54:06 2002 +++ SMTPDirect.py Fri Sep 6 11:17:53 2002 @@ -44,8 +44,15 @@ +def throttle(number_of_recipients): + # or some algorithm to compute inter-connection delay + delay = int(number_of_recipients / 2000) + return delay + + def process(mlist, msg, msgdata): recips = msgdata.get('recips') + throttle_delay = throttle(len(recips)) if not recips: # Nobody to deliver to! return @@ -72,6 +79,8 @@ else: for chunk in chunks: deliver(admin, msgtext, chunk, refused) + if throttle_delay: + time.sleep(throttle_delay) finally: t1 = time.time() mlist.Lock() cut here------------------------------------------------------ At 16:12 05/09/2002 -0600, Kory Wheatley wrote:
Using Mailman 2.0.13 on Red Hat 7.2
I've got a Mailman mailing list that contains 14,000 subscribers, on a Linux box that's using sendmail has the delivery agent that sends all of the messages to one mail host, because all 14,000 subscribers have an email account on our mail server. This Linux box is only used to deliver our Mailman mailing to our Mail server which is a HPUX 11 server, using sendmail as the routing agent.
We sent a message to the 14,000 recipient Mailman mailing list the other day and it put the load on our Unix Box that contains our mail server to the roof
What I have done to maybe slow down the delivery to sendmail that creates the queue files then delivers them to our mail server is in the Mailman SMTPDirect.py I have used the sleep command to sleep after delivering so that our mail server can catch up before the next batch is delivered, this way its not being bombarded.
Now here is where I would appreciate help, I'm not a experienced Python programmer and maybe what I did below is stupid and will not work (using the time.sleep(25) command), but please look at the code and if it will not work please let me know what I can code that will work that will slow down the delivery to our mail server.
------------CODE--------------------
def deliver(envsender, msgtext, recips, failures): refused = {} # Gather statistics on how long each SMTP dialog takes. ## t0 = time.time() try: conn = smtplib.SMTP(mm_cfg.SMTPHOST, mm_cfg.SMTPPORT) try: # make sure the connect happens, which won't be done by the # constructor if SMTPHOST is false time.sleep(25) refused = conn.sendmail(envsender, recips, msgtext) finally: ## t1 = time.time() ## syslog('smtp', 'smtp for %d recips, completed in %.3f seconds' % ## (len(recips), (t1-t0))) conn.quit() except smtplib.SMTPRecipientsRefused, e: refused = e.recipients # MTA not responding, or other socket problems, or any other kind of
# SMTPException. In that case, nothing got delivered except (socket.error, smtplib.SMTPException), e:
------------END-----------------
_______________________________________________ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman-21/listinfo/mailman-developers