Re: [Mailman-Developers] Mailman password and administrative reminders
? This very message came to me with the following header:
Errors-To: mailman-developers-admin@python.org
All my bounces come to the list admin address, set in the admin webpage, in the second field of General Options. Do you have that set to something, and bounces still come to root?
- Bounces are sent to the poor postmaster instead of a -admin address. I'm not entirely certain, but I think an Errors-To: header or something like that in all Mailman messages might allow one to distribute that load somewhat.
Errors-To: mailman-developers-admin@python.org
All my bounces come to the list admin address, set in the admin webpage, in the second field of General Options. Do you have that set to something, and bounces still come to root?
- Bounces are sent to the poor postmaster instead of a -admin address. I'm not entirely certain, but I think an Errors-To: header or something like that in all Mailman messages might allow one to distribute that load somewhat.
Errors-To: is way deprecated anyhow... It was intended to supercede the envelope From address for errors, but it violated RFC1123 and was subsequently ignored by many MTAs.
Chris
Christopher Lindsey, Senior System Engineer National Center for Supercomputing Applications (NCSA)
Hmm....
It is the administrative messages that don't have this set, I believe. We had a large list (4500 members) that was just converted from majordomo. Our password reminders went out, and about 500 bounces came to postmaster instead of going to the list admin. We're having the auto-bounce feature turn people off, but the list has only been broadcast once, so people haven't been auto-unsubscribed yet (a more agressive unsubcribe might be helpful for these once-a-month broadcast lists, but perhaps I don't have it tuned right.)
The message actually sent to the list (ie, the non-administrative stuff) seemed to work properly and bounce to the list owners.
BTW - minor bug. Even if you turn password reminders off, the welcome message that's created still says they are sent out.
David.
** WARNING ** This message was composed by a temporarily crippled programmer using a speech recognition system. Please don't be alarmed if some extremely strange word usage shows up in the message. Many thanks....
On Wed, 1 Mar 2000, Dan Mick wrote:
? This very message came to me with the following header:
Errors-To: mailman-developers-admin@python.org
All my bounces come to the list admin address, set in the admin webpage, in the second field of General Options. Do you have that set to something, and bounces still come to root?
- Bounces are sent to the poor postmaster instead of a -admin address. I'm not entirely certain, but I think an Errors-To: header or something like that in all Mailman messages might allow one to distribute that load somewhat.
Here's the headers for the reminder:
Return-Path: mailman-owner@chelsea.net
Received: from killian.chelsea.net (localhost [127.0.0.1])
by killian.chelsea.net (8.9.3/8.9.3) with ESMTP id FAA15246
for donad@gaigo.sundai.ac.jp; Wed, 1 Mar 2000 05:52:23 -0500 (EST)
Date: Wed, 1 Mar 2000 05:52:23 -0500 (EST)
Message-Id: 200003011052.FAA15246@killian.chelsea.net
From: mailman-owner@chelsea.net
Subject: thebody.com mailing list memberships reminder
To: ###A USER###
X-No-Archive: yes
Precedence: bulk
X-Mailman-Version: 1.1
Precedence: bulk
List-Id: The Test List
It should be from (I think), thebody-admin@thebody.com, with an Errors-To: header.
Not sure why List-Id: is The Test List...that's odd.
David.
** WARNING ** This message was composed by a temporarily crippled programmer using a speech recognition system. Please don't be alarmed if some extremely strange word usage shows up in the message. Many thanks....
On Wed, 1 Mar 2000, Dan Mick wrote:
? This very message came to me with the following header:
Errors-To: mailman-developers-admin@python.org
All my bounces come to the list admin address, set in the admin webpage, in the second field of General Options. Do you have that set to something, and bounces still come to root?
- Bounces are sent to the poor postmaster instead of a -admin address. I'm not entirely certain, but I think an Errors-To: header or something like that in all Mailman messages might allow one to distribute that load somewhat.
Here's the problem in a nutshell (if my quick browse of the thread is correct).
When a user is to get the password reminder, Mailman collates the passwords for all the lists that the user is on (for that virtual domain, but let's ignore that). So one password reminder refers to potentially several lists. So which list -owner address (e.g. bounce detecting addr) should get the bounces?
As I see it, the right solution is the following:
Mailman has no catch-all address like Majordomo has. I.e. you can't send `help' to mailman@... unless you actually craft a normal Mailman list for this addr. This is bogus, because it just thinks it's a normal list, not something special. Step one of the fix is to write scripts that can handle these over-arching addrs. Then of course, we'd make mailman-owner@... the recipient of all the bounced password reminders. The script on the tail of that would Do The Right Thing.
Unfortunately, the correct solution, IMO requires user databases, because otherwise you need to cycle through all the lists looking for the user address to disable. Imagine for a moment, hundreds of bounces coming back starting at 12:02 the first of every month, each one trying to hit every list on your site. Ouch!
I've been seriously thinking about adding support for the first bullet for 1.2 (scratching my own itch, doncha know), but I also just want to start getting the betas out, so it may have to wait. Harald's got stuff in the pipeline to support user databases, but that's defiitely a post 1.2 feature.
If you wanted to play with this stuff in the meantime, you could implement #1, and see how bad a hit the touch-all-lists approach would be.
-Barry
Perhaps I'm missing something at the fundamental architecture level, but wouldn't it make more sense to just cycle through each list and send the reminder using the list owner? It's probably nice to only get one password reminder email, but how big and inconvenience would that be, anyway?
This way, each user gets one email per list, and if it bounces, you know what list to take it off of. That won't disable the address server wide, but sooner or later some bounces will happen on the other lists and take care of it. This way you don't have to do the user database thing.
I have nothing more than anecdotal evidence (ie, my ISP), but how often is one address on multiple lists at a given site?
We've had to turn off password notification, because now all the people on this list who got password notifications are now replying to mailman-owner with list related requests. Sigh....
David.
** WARNING ** This message was composed by a temporarily crippled programmer using a speech recognition system. Please don't be alarmed if some extremely strange word usage shows up in the message. Many thanks....
On Thu, 2 Mar 2000, Barry A. Warsaw wrote:
Here's the problem in a nutshell (if my quick browse of the thread is correct).
When a user is to get the password reminder, Mailman collates the passwords for all the lists that the user is on (for that virtual domain, but let's ignore that). So one password reminder refers to potentially several lists. So which list -owner address (e.g. bounce detecting addr) should get the bounces?
As I see it, the right solution is the following:
Mailman has no catch-all address like Majordomo has. I.e. you can't send `help' to mailman@... unless you actually craft a normal Mailman list for this addr. This is bogus, because it just thinks it's a normal list, not something special. Step one of the fix is to write scripts that can handle these over-arching addrs. Then of course, we'd make mailman-owner@... the recipient of all the bounced password reminders. The script on the tail of that would Do The Right Thing.
Unfortunately, the correct solution, IMO requires user databases, because otherwise you need to cycle through all the lists looking for the user address to disable. Imagine for a moment, hundreds of bounces coming back starting at 12:02 the first of every month, each one trying to hit every list on your site. Ouch!
I've been seriously thinking about adding support for the first bullet for 1.2 (scratching my own itch, doncha know), but I also just want to start getting the betas out, so it may have to wait. Harald's got stuff in the pipeline to support user databases, but that's defiitely a post 1.2 feature.
If you wanted to play with this stuff in the meantime, you could implement #1, and see how bad a hit the touch-all-lists approach would be.
-Barry
In message 14526.29365.432370.248009@anthem.cnri.reston.va.us, "Barry A. Wars aw" writes:
Here's the problem in a nutshell (if my quick browse of the thread is correct).
When a user is to get the password reminder, Mailman collates the passwords for all the lists that the user is on (for that virtual domain, but let's ignore that). So one password reminder refers to potentially several lists. So which list -owner address (e.g. bounce detecting addr) should get the bounces?
What about sending the messages out with multiple addresses in the From header? It's a little weird, but is RFC legal, and that way each reminder will bounce to all of the lists whose password is listed in the email.
-- Ted Cabeen http://www.pobox.com/~secabeen secabeen@pobox.com Check Website or finger for PGP Public Key secabeen@midway.uchicago.edu "I have taken all knowledge to be my province." -F. Bacon cococabeen@aol.com "Human kind cannot bear very much reality."-T.S.Eliot 73126.626@compuserve.com
"DB" == David Birnbaum davidb@chelsea.net writes:
DB> I have nothing more than anecdotal evidence (ie, my ISP), but
DB> how often is one address on multiple lists at a given site?It all depends. Some sites might run only a few, or mostly unconnected lists. At Python.Org, such a change would cause a revolt :).
DB> We've had to turn off password notification, because now all
DB> the people on this list who got password notifications are now
DB> replying to mailman-owner with list related requests. Sigh....Yup, I agree, the current situation is suboptimal. It will be fixed, it's just a matter of when.
-Barry
Revolt among the Python developers would certainly be a bad thing, given they are doing the development ;)
Thanks for the help, looking forward to eventual solution.
David.
** WARNING ** This message was composed by a temporarily crippled programmer using a speech recognition system. Please don't be alarmed if some extremely strange word usage shows up in the message. Many thanks....
On Thu, 2 Mar 2000 bwarsaw@cnri.reston.va.us wrote:
"DB" == David Birnbaum davidb@chelsea.net writes:
DB> I have nothing more than anecdotal evidence (ie, my ISP), but DB> how often is one address on multiple lists at a given site?It all depends. Some sites might run only a few, or mostly unconnected lists. At Python.Org, such a change would cause a revolt :).
DB> We've had to turn off password notification, because now all DB> the people on this list who got password notifications are now DB> replying to mailman-owner with list related requests. Sigh....Yup, I agree, the current situation is suboptimal. It will be fixed, it's just a matter of when.
-Barry
Howdy, I just had a list corruption today - twice - with no smoking gun in the log files that indicated the cause of the problem, just that the list database became corrupted (see error messages below). It's a list of about 22,000 people at the moment. I was able to dump the list members and list configuration, delete the list, and recreate it. However, it became corrupted again a few hours later, although not as badly. In the first corruption, the list was unusable by the web interface. The second time, it "fixed" itself, in that the web interface started working again. I investigated further, and I discovered a lot of broken locks (which seems odd, given a lifetime of five hours by default): Aug 13 08:44:14 2003 (911) thebody.lock lifetime has expired, breaking Aug 13 11:14:07 2003 (911) thebody.lock lifetime has expired, breaking Aug 13 12:40:41 2003 (911) thebody.lock lifetime has expired, breaking Aug 13 15:11:50 2003 (911) thebody.lock lifetime has expired, breaking Aug 13 15:58:56 2003 (911) thebody.lock lifetime has expired, breaking Aug 13 16:26:56 2003 (911) thebody.lock lifetime has expired, breaking Not sure if this has anything to do with it, but it's the only unusual thing. I also noted that the dates on the locks are often in the future; perhaps this is by design, it's certainly very strange. The locks are appearing/disappearing appropriately, as far as I can tell, and nothing strange is showing up otherwise. I've stopped and restarted the master mailman process, and dumped/restored the list again, and all appears well. Any suggestions on where to start? I'm running the latest mailman 2.1.2, on Solaris 2.8, with Python 2.2.2. Haven't had any problems with the machine or memory that have showed up in the logs. Thanks in advance, David. ----- Aug 13 17:01:40 2003 (22680) couldn't load config file /home/mailman/lists/thelist/config.pck Aug 13 17:01:40 2003 (22680) couldn't load config file /home/mailman/lists/thelist/config.pck.last Aug 13 17:01:40 2003 (22680) couldn't load config file /home/mailman/lists/thelist/config.db [Errno 2] No such file or directory: '/home/mailman/lists/thelist/config.db' Aug 13 17:01:40 2003 (22680) couldn't load config file /home/mailman/lists/thelist/config.db.last [Errno 2] No such file or directory: '/home/mailman/lists/thelist/config.db.last' Aug 13 17:01:40 2003 (22680) All thelist fallbacks were corrupt, giving up Aug 13 17:01:40 2003 admin(22680): @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ admin(22680): [----- Mailman Version: 2.1.2 -----] admin(22680): [----- Traceback ------] admin(22680): Traceback (most recent call last): admin(22680): File "/home/mailman/scripts/driver", line 87, in run_main admin(22680): main() admin(22680): File "/home/mailman/Mailman/Cgi/admin.py", line 162, in main admin(22680): mlist.Lock() admin(22680): File "/home/mailman/Mailman/MailList.py", line 159, in Lock admin(22680): self.Load() admin(22680): File "/home/mailman/Mailman/MailList.py", line 598, in Load admin(22680): raise Errors.MMCorruptListDatabaseError, e admin(22680): MMCorruptListDatabaseError: [Errno 2] No such file or directory: '/home/mailman/lists/ thelist/config.db.last' admin(22680): [----- Python Information -----] admin(22680): sys.version = 2.2.2 (#1, Feb 4 2003, 14:15:12) [GCC 3.2.1] admin(22680): sys.executable = /usr/local/bin/python admin(22680): sys.prefix = /opt/python/2.2.2 admin(22680): sys.exec_prefix = /opt/python/2.2.2 admin(22680): sys.path = /opt/python/2.2.2 admin(22680): sys.platform = sunos5 admin(22680): [----- Environment Variables -----] admin(22680): HTTP_COOKIE: thelist+admin=28020000006988523a3f7328000000633165626261613432313637316 66631633863613437326636386435353365623233326666623938 admin(22680): SERVER_SOFTWARE: Apache/1.3.26 (Unix) mod_ssl/2.8.10 OpenSSL/0.9.6c admin(22680): PYTHONPATH: /home/mailman admin(22680): SCRIPT_FILENAME: /home/mailman/cgi-bin/admin.cgi admin(22680): SERVER_ADMIN: webmaster@chelsea.net admin(22680): SCRIPT_NAME: /cgi-bin/admin.cgi admin(22680): REQUEST_METHOD: GET admin(22680): HTTP_HOST: mailman.chelsea.net admin(22680): PATH_INFO: /thelist admin(22680): SERVER_PROTOCOL: HTTP/1.1 admin(22680): QUERY_STRING: admin(22680): TZ: US/Eastern admin(22680): REQUEST_URI: /cgi-bin/admin.cgi/thelist admin(22680): HTTP_ACCEPT: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/msword , application/vnd.ms-excel, application/vnd.ms-powerpoint, application/x-shockwave-flash, */* admin(22680): HTTP_USER_AGENT: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0) admin(22680): HTTP_CONNECTION: Keep-Alive admin(22680): SERVER_NAME: mailman.chelsea.net admin(22680): REMOTE_ADDR: 209.212.72.130 admin(22680): REMOTE_PORT: 5086 admin(22680): HTTP_ACCEPT_LANGUAGE: en-us admin(22680): PATH_TRANSLATED: /home/mailman/htdocs/thelist admin(22680): SERVER_PORT: 80 admin(22680): GATEWAY_INTERFACE: CGI/1.1 admin(22680): HTTP_ACCEPT_ENCODING: gzip, deflate admin(22680): SERVER_ADDR: 209.212.66.37
Folks, I found the problem, after grabbing a truss of the web process. It turns out the resident process size became 30M, which turned out to be the RLimitMEM on our web server by default. It was sporadic because the list was *just* under the limit, and it would only fail if it couldn't lock the list quickly enough (minor memory leak there, perhaps?) Since python grabs the Err#12 ENOMEM, it would be nice to propagate that one back if possible! Sorry for the false alarm! Side question, though - why *ARE* the locks made "in the future"? Thanks, David. ----- On Wed, 13 Aug 2003, David Birnbaum wrote:
Howdy,
I just had a list corruption today - twice - with no smoking gun in the log files that indicated the cause of the problem, just that the list database became corrupted (see error messages below). It's a list of about 22,000 people at the moment.
I was able to dump the list members and list configuration, delete the list, and recreate it. However, it became corrupted again a few hours later, although not as badly. In the first corruption, the list was unusable by the web interface. The second time, it "fixed" itself, in that the web interface started working again. I investigated further, and I discovered a lot of broken locks (which seems odd, given a lifetime of five hours by default):
Aug 13 08:44:14 2003 (911) thebody.lock lifetime has expired, breaking Aug 13 11:14:07 2003 (911) thebody.lock lifetime has expired, breaking Aug 13 12:40:41 2003 (911) thebody.lock lifetime has expired, breaking Aug 13 15:11:50 2003 (911) thebody.lock lifetime has expired, breaking Aug 13 15:58:56 2003 (911) thebody.lock lifetime has expired, breaking Aug 13 16:26:56 2003 (911) thebody.lock lifetime has expired, breaking
Not sure if this has anything to do with it, but it's the only unusual thing. I also noted that the dates on the locks are often in the future; perhaps this is by design, it's certainly very strange. The locks are appearing/disappearing appropriately, as far as I can tell, and nothing strange is showing up otherwise.
I've stopped and restarted the master mailman process, and dumped/restored the list again, and all appears well. Any suggestions on where to start? I'm running the latest mailman 2.1.2, on Solaris 2.8, with Python 2.2.2. Haven't had any problems with the machine or memory that have showed up in the logs.
Thanks in advance,
David.
-----
Aug 13 17:01:40 2003 (22680) couldn't load config file /home/mailman/lists/thelist/config.pck Aug 13 17:01:40 2003 (22680) couldn't load config file /home/mailman/lists/thelist/config.pck.last Aug 13 17:01:40 2003 (22680) couldn't load config file /home/mailman/lists/thelist/config.db [Errno 2] No such file or directory: '/home/mailman/lists/thelist/config.db' Aug 13 17:01:40 2003 (22680) couldn't load config file /home/mailman/lists/thelist/config.db.last [Errno 2] No such file or directory: '/home/mailman/lists/thelist/config.db.last' Aug 13 17:01:40 2003 (22680) All thelist fallbacks were corrupt, giving up Aug 13 17:01:40 2003 admin(22680): @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ admin(22680): [----- Mailman Version: 2.1.2 -----] admin(22680): [----- Traceback ------] admin(22680): Traceback (most recent call last): admin(22680): File "/home/mailman/scripts/driver", line 87, in run_main admin(22680): main() admin(22680): File "/home/mailman/Mailman/Cgi/admin.py", line 162, in main admin(22680): mlist.Lock() admin(22680): File "/home/mailman/Mailman/MailList.py", line 159, in Lock admin(22680): self.Load() admin(22680): File "/home/mailman/Mailman/MailList.py", line 598, in Load admin(22680): raise Errors.MMCorruptListDatabaseError, e admin(22680): MMCorruptListDatabaseError: [Errno 2] No such file or directory: '/home/mailman/lists/ thelist/config.db.last' admin(22680): [----- Python Information -----] admin(22680): sys.version = 2.2.2 (#1, Feb 4 2003, 14:15:12) [GCC 3.2.1] admin(22680): sys.executable = /usr/local/bin/python admin(22680): sys.prefix = /opt/python/2.2.2 admin(22680): sys.exec_prefix = /opt/python/2.2.2 admin(22680): sys.path = /opt/python/2.2.2 admin(22680): sys.platform = sunos5 admin(22680): [----- Environment Variables -----] admin(22680): HTTP_COOKIE: thelist+admin=28020000006988523a3f7328000000633165626261613432313637316 66631633863613437326636386435353365623233326666623938 admin(22680): SERVER_SOFTWARE: Apache/1.3.26 (Unix) mod_ssl/2.8.10 OpenSSL/0.9.6c admin(22680): PYTHONPATH: /home/mailman admin(22680): SCRIPT_FILENAME: /home/mailman/cgi-bin/admin.cgi admin(22680): SERVER_ADMIN: webmaster@chelsea.net admin(22680): SCRIPT_NAME: /cgi-bin/admin.cgi admin(22680): REQUEST_METHOD: GET admin(22680): HTTP_HOST: mailman.chelsea.net admin(22680): PATH_INFO: /thelist admin(22680): SERVER_PROTOCOL: HTTP/1.1 admin(22680): QUERY_STRING: admin(22680): TZ: US/Eastern admin(22680): REQUEST_URI: /cgi-bin/admin.cgi/thelist admin(22680): HTTP_ACCEPT: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/msword , application/vnd.ms-excel, application/vnd.ms-powerpoint, application/x-shockwave-flash, */* admin(22680): HTTP_USER_AGENT: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0) admin(22680): HTTP_CONNECTION: Keep-Alive admin(22680): SERVER_NAME: mailman.chelsea.net admin(22680): REMOTE_ADDR: 209.212.72.130 admin(22680): REMOTE_PORT: 5086 admin(22680): HTTP_ACCEPT_LANGUAGE: en-us admin(22680): PATH_TRANSLATED: /home/mailman/htdocs/thelist admin(22680): SERVER_PORT: 80 admin(22680): GATEWAY_INTERFACE: CGI/1.1 admin(22680): HTTP_ACCEPT_ENCODING: gzip, deflate admin(22680): SERVER_ADDR: 209.212.66.37
I'm not ignoring you guys. ;) Well, actually I am, but I wish I didn't have to. I'm slammed to the gills with work stuff these days. I had hope to have some free evenings this week to catch up, but no dice.
Chuq, I have a working patch for you that I still haven't had time to check in. I'm also nearly finished with a patch that should make bounce processing much less painful. Hopefully, I'll have a few hours soon to finish that up and then release a MM2.1.3.
Sigh. I'll be back soon. -Barry
participants (8)
-
Barry A. Warsaw -
Barry Warsaw -
bwarsaw＠cnri.reston.va.us
-
Christopher Lindsey -
Dan Mick -
David Birnbaum -
David Birnbaum -
Ted Cabeen