[Mailman-Developers] big list
Fil
fil@rezo.net
Mon, 18 Mar 2002 00:44:04 +0100
@ Barry A. Warsaw <barry@zope.com> :
> >>>>> "F" == Fil <fil@rezo.net> writes:
> F> I am ashamed: "cvs update" did me no good; some scripts were
> F> up-to-date, others definitely not. Now everything's a clean
> F> install, I'll let the week-end go without a post to the
> F> -developpers list.
>
> :)
Hi there, my week end's over!
Here are tracebacks of bugs observed with Mailman 2.1b1 (I'm positive this
is a nice and complete install from cvs!) over the week end.
Two bugs seem to be running:
1) File "/home/mailman/Mailman/MailList.py", line 980, in ProcessConfirmation
addr = userdesc.address
AttributeError: address
One of my lists' admin (his list has 13 000 subscribers) has hit this one
several times this weekend (he received the tracebacks by email).
2) A timeout occurs on the 'pending' database and maybe on the list database
also when we send many messages. From observing the processes running on the
server and the flow of emails, it looks like the culprit is the
BounceRunner, which locks these databases for long periods, and does not
release the locks fast enough for the other processes.
Maybe this is because the lists are big (ie long time
opening/saving/checking if the bounce is a member). We don't care much about
the BounceRunner losing its lock, but we should let it block the other
processes, who are responding to the users (be it the Web interface or the
mail command runner).
On my setup, a bounce on the 50 000 subscribers' list is processed in about
3-4 seconds. The BounceRunner takes up 100% of one CPU (I have two) and a
lot of memory (although less than last time I raised that issue).
But count: 3 seconds * 40 000 bounces = 33 hours. So if someone migrates her
big list from some bounce-blind MLM to Mailman, she'll run into that issue:
sub/unsub/listinfo/whatever requests on the list returning exceptions for 33
hours.
This has happened many times also over the week end.
Hope this can be helpful.
SOME TRACEBACKS:
----- Original Message -----
From: <listname-admin@rezo.net>
To: <listname-admin@rezo.net>
Sent: Sunday, March 17, 2002 2:09 PM
Subject: Erreur Mailman inattendue
Une erreur Mailman inattendue s'est produite dans
MailCommandHandler.ParseMailCommand(). En voici les raisons:
Traceback (most recent call last):
File "/home/mailman/Mailman/MailCommandHandler.py", line 273, in
ParseMailCommands
self.__dispatch[cmd](args, line, msg)
File "/home/mailman/Mailman/MailCommandHandler.py", line 727, in
ProcessConfirmCmd
results = self.ProcessConfirmation(args[0], msg)
File "/home/mailman/Mailman/MailList.py", line 980, in ProcessConfirmation
addr = userdesc.address
AttributeError: address
Pour gerer votre abonnement :
http://listes.rezo.net/mailman/listinfo/listname
The error log says about the same thing:
Mar 17 14:09:04 2002 (9618) Unexpected Mailman error:
Traceback (most recent call last):
File "/home/mailman/Mailman/MailCommandHandler.py", line 273, in ParseMailCom
self.__dispatch[cmd](args, line, msg)
File "/home/mailman/Mailman/MailCommandHandler.py", line 727, in ProcessConfi
results = self.ProcessConfirmation(args[0], msg)
File "/home/mailman/Mailman/MailList.py", line 980, in ProcessConfirmation
addr = userdesc.address
AttributeError: address
***********************************************************************
Mar 17 18:52:14 2002 admin(8563):
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
admin(8563): [----- Mailman Version: 2.1b1 -----]
admin(8563): [----- Traceback ------]
admin(8563): Traceback (most recent call last):
admin(8563): File "/home/mailman/scripts/driver", line 82, in run_main
admin(8563): main()
admin(8563): File "/home/mailman/Mailman/Cgi/subscribe.py", line 94, in main
admin(8563): process_form(mlist, doc, cgidata, language)
admin(8563): File "/home/mailman/Mailman/Cgi/subscribe.py", line 176, in proc
admin(8563): mlist.AddMember(userdesc, remote)
admin(8563): File "/home/mailman/Mailman/MailList.py", line 711, in AddMember
admin(8563): cookie = Pending.new(Pending.SUBSCRIPTION, userdesc)
admin(8563): File "/home/mailman/Mailman/Pending.py", line 61, in new
admin(8563): lock.lock(timeout=30)
admin(8563): File "/home/mailman/Mailman/LockFile.py", line 292, in lock
admin(8563): raise TimeOutError
admin(8563): TimeOutError
***********************************************************************
Mar 17 19:11:25 2002 (9618) Unexpected Mailman error:
Traceback (most recent call last):
File "/home/mailman/Mailman/MailCommandHandler.py", line 273, in
ParseMailCom
self.__dispatch[cmd](args, line, msg)
File "/home/mailman/Mailman/MailCommandHandler.py", line 727, in
ProcessConfi
results = self.ProcessConfirmation(args[0], msg)
File "/home/mailman/Mailman/MailList.py", line 968, in ProcessConfirmation
data = Pending.confirm(cookie)
File "/home/mailman/Mailman/Pending.py", line 94, in confirm
lock.lock(timeout=30)
File "/home/mailman/Mailman/LockFile.py", line 292, in lock
raise TimeOutError
TimeOutError
***********************************************************************
Mar 17 20:48:03 2002 admin(22240):
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
admin(22240): [----- Mailman Version: 2.1b1 -----]
admin(22240): [----- Traceback ------]
admin(22240): Traceback (most recent call last):
admin(22240): File "/home/mailman/scripts/driver", line 82, in run_main
admin(22240): main()
admin(22240): File "/home/mailman/Mailman/Cgi/options.py", line 164, in main
admin(22240): mlist.MailUserPassword(user)
admin(22240): File "/home/mailman/Mailman/Deliverer.py", line 107, in MailUse
admin(22240): {'user' : cpuser,
admin(22240): KeyError: Rxxxxxxxxxxxx@AOL.COM
admin(22240): [----- Python Information -----]
admin(22240): sys.version = 2.1 (#1, Jun 1 2001, 23:51:21)
[GCC 2.95.3 20010125 (prerelease)]
admin(22240): sys.executable = /usr/local/bin/python2.1
admin(22240): sys.prefix = /usr/local
admin(22240): sys.exec_prefix = /usr/local
admin(22240): sys.path = /usr/local
admin(22240): sys.platform = linux2
admin(22240): [----- Environment Variables -----]
admin(22240): DOCUMENT_ROOT: /var/www/listes.rezo.net
admin(22240): SERVER_ADDR: 80.67.170.17
admin(22240): HTTP_ACCEPT_ENCODING: gzip, deflate
admin(22240): CONTENT_LENGTH: 54
admin(22240): CONTENT_TYPE: application/x-www-form-urlencoded
admin(22240): PATH_TRANSLATED: /var/www/listes.rezo.net/courrier-balkans
admin(22240): GATEWAY_INTERFACE: CGI/1.1
admin(22240): UNIQUE_ID: PJTy81BDqhEAAEPnHyQ
admin(22240): HTTP_ACCEPT_LANGUAGE: fr
admin(22240): REMOTE_ADDR: 19xxxxxxxxxxxxx
admin(22240): SERVER_PORT: 80
admin(22240): HTTP_USER_AGENT: Mozilla/4.0 (compatible; MSIE 6.0; AOL 7.0; Wi
admin(22240): HTTP_ACCEPT: image/gif, image/x-xbitmap, image/jpeg, image/pjpe
admin(22240): REQUEST_URI: /mailman/options/courrier-balkans
admin(22240): QUERY_STRING:
admin(22240): SCRIPT_FILENAME: /home/mailman/cgi-bin/options
admin(22240): SCRIPT_URL: /mailman/options/courrier-balkans
admin(22240): HTTP_HOST: listes.rezo.net
admin(22240): REQUEST_METHOD: POST
admin(22240): SERVER_SIGNATURE:
admin(22240): SCRIPT_URI: http://listes.rezo.net/mailman/options/courrier-bal
admin(22240): SCRIPT_NAME: /mailman/options
admin(22240): SERVER_ADMIN: fil@rezo.net
admin(22240): SERVER_SOFTWARE: Apache/1.3.23 (Unix) Debian GNU/Linux mod_fast
admin(22240): PYTHONPATH: /home/mailman
admin(22240): PATH_INFO: /courrier-balkans
admin(22240): HTTP_REFERER: http://listes.rezo.net/mailman/options/courrier-b
admin(22240): HTTP_PRAGMA: no-cache
admin(22240): SERVER_NAME: listes.rezo.net
-- Fil