Gergely Madarasz writes:

I have problems with mailpasswds... first it did not work because of this line: url = list.GetOptionsURL(user) I changed it to: url = list.GetAbsoluteOptionsURL(user) Now it starts (the text is a bit misplaced in the mails it sends), but forks too many times (I have two lists, 500 and 120 subscribers): [...] There should be a better way to handle these mails... either to have a better mailing algorithm, or to handle this error, wait for a minute, then continue.

You don't say which version you're running, but we ran into the same problem here at python.org (yesterday, when the passwords went out), and i've attacked the problem on two fronts in the current code. I'll describe what i did for the first one, if you want to try to reproduce it in your own code, and include a new version of cron/mailpasswds for 1.0b4 for the second.

One fix is to change the "os.forks()" in the scripts/deliver script to use the following 'forker()' routine, which recognizes errno.EAGAIN, and retries to fork a specified number of times. (You have to have settings in the module for TRIES and REFRACT - i use 5 and 15, respectively.)

TRIES = 5 REFRACT = 15

def forker(tries=TRIES, refract=REFRACT): """Fork, retrying on EGAIN errors with refract secs pause between tries.

Returns value of os.fork(), or raises the exception for:
 (1) non-EAGAIN exception, or
 (2) EGAIN exception encountered more than tries times."""
got = 0
for i in range(tries):
    # Loop until we successfully fork or the number tries is exceeded.
    try:
        got = os.fork()
        break
    except os.error, val:
        import errno, sys, time
        if val[0] == errno.EAGAIN:
            # Resource temporarily unavailable.
            time.sleep(refract)
        else:
            # No go - reraise original exception, same stack frame and all.
            raise val, None, sys.exc_info()[2]
return got

Another solution is to have the mailpasswds script take some time to do an os.wait() every several users - i'm attaching a version of cron/mailpasswds for 1.0b4 that does this.

#! /usr/bin/env python # # Copyright (C) 1998 by the Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.

"""Send password reminders for all lists to all users.

We accumulate users and their passwords, and use the last list to send a single message to each user with their complete collection of passwords, rather than sending a single message for each password."""

# This puppy should probably do lots of logging.

import sys, os, string, time, errno import paths import maillist, mm_cfg, mm_message, mm_utils

# Give time for the delivery process-forks to clear every so often, to # avoid saturation of the process table. Set zero or negative for no # pauses. PAUSE_FREQUENCY = 20

USERPASSWORDSTEXT = """ This is a reminder, sent out once a month, about your %s mailing list memberships. It includes your subscription info and how to use it to change it or unsubscribe from a list.

Passwords for %s:

%s %s You can visit the URLs to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.

In addition to the URL interfaces, you can also use email to make such changes. For more info, send a message to the '-request' address of the list (for example, %s-request@%s) containing just the word 'help' in the message body, and an email message will be sent to you with instructions.

If you have questions, problems, comments, etc, send them to mailman-owner@%s. Thanks! """

def MailAllPasswords(list, users): """Send each user their complete list of passwords.

The list can be any random one - it is only used for the message
delivery mechanism."""
subj = '%s maillist memberships reminder\n' % list.host_name
count = PAUSE_FREQUENCY
for user, data in users.items():
table = []
for l, p, u in data:
    if len(l) > 9:
	table.append("%s\n           %-10s\n%s\n" % (l, p, u))
    else:
	table.append("%-10s %-10s\n%s\n" % (l, p, u))
header = ("%-10s %-10s\n%-10s %-10s"
	  % ("List", "Password // URL", "----", "--------"))
text = USERPASSWORDSTEXT % (list.host_name,
                                user,
			    header,
			    string.join(table, "\n"),
			    l, list.host_name,
			    list.host_name)
list.SendTextToUser(subject = subj,
		    recipient = user,
		    text = text,
		    sender = mm_cfg.MAILMAN_OWNER,
                        add_headers = ["X-No-Archive: yes"],
                        raw=1)
    count = count - 1
    if count == 0:
        # The pause that refreshes.
        waitall()
        count = PAUSE_FREQUENCY

def main(): """Consolidate all the list/url/password info for each user, so we send the user a single message with the info for all their lists on this site.""" list = None users = {} # user: (listname, password, url) for name in mm_utils.list_names(): list = maillist.MailList(name) list_name = list.real_name reminders_to_admins = list.reminders_to_admins for user, password in list.passwords.items(): url = list.GetAbsoluteOptionsURL(user) if reminders_to_admins: recipient = "%s-admin@%s" % tuple(string.split(user, '@')) else: recipient = user if users.has_key(recipient): users[recipient].append(list_name, password, url) else: users[recipient] = [(list_name, password, url)] # Unlocking each list after identifying passwords, but before having # the consolidated list, means that there is a window for discrepancy # between the reported and actual password. Big deal - if the user # changed the password in the meanwhile, they'll realize it, and it's # not worth the extra deadlock risk. list.Unlock()

if list:
MailAllPasswords(list, users)

def waitall(): """Return only when there are no forked subprocesses running.""" try: while 1: os.wait() except os.error, val: if val[0] == errno.ECHILD: # errno.ECHILD: "No child processes" return else: raise val, None, sys.exc_info()[2]

if __name__ == "__main__": main()

I've done a little testing of both these approaches, but since the problem exists at the resource limit, and entails sending out messages to lots of people, i'm limited in the testing i could do. Caveat emptor! (And "everyone scrutinize" - the more eyes the better.)

Ken Manheimer klm@python.org 703 620-8990 x268 (orporation for National Research |nitiatives

# If you appreciate Python, consider joining the PSA! #
	  # <http://www.python.org/psa/>. #