Encoding problem: Mailman bails during "Administrative requests"
I asked the Mailman-Users list this question last week, but got no useful replies. Since this is a bug in the code, someone here might know how to proceed. I might add to the original question: Would it help to go to the very latest Mailman release? RRRRRRRRRRRRRRRRRRRRRRRR I'd like some help on dealing with this problem permanently. I've been getting this behavior every couple of months and haven't found a good solution yet. The way it happens is that I'm on the "Administrative requests" page for some mailing list and when I submit the deletion requests I get the following reply from the browser: Bug in Mailman version 2.1.6 We're sorry, we hit a bug! Please inform the webmaster for this site of this problem. Printing of traceback and other system information has been explicitly inhibited, but the webmaster can find this information in the Mailman error logs. Usually you can then go back an delete individual entries, but there will be one or more that produce the error, with this sort of traceback (below): I take it that the email in question (SPAM email) isn't properly encoded. Any idea on how to get around this problem? In the past I've gone into the queues and deleted files, etc, but the process isn't easy, and of course I won't get any cooperation from the SPAMers sending the messages. The OS is FreeBSD 4.9 and Mailman is 2.1.6_1, built from source. ========== TRACEBACK START Dec 11 13:13:52 2005 admin(10012): @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ admin(10012): [----- Mailman Version: 2.1.6 -----] admin(10012): [----- Traceback ------] admin(10012): Traceback (most recent call last): admin(10012): File "/usr/local/mailman/scripts/driver", line 101, in run_main admin(10012): main() admin(10012): File "/usr/local/mailman/Mailman/Cgi/admindb.py", line 163, in main admin(10012): process_form(mlist, doc, cgidata) admin(10012): File "/usr/local/mailman/Mailman/Cgi/admindb.py", line 713, in process_form admin(10012): forward, forwardaddr) admin(10012): File "/usr/local/mailman/Mailman/ListAdmin.py", line 167, in HandleRequest admin(10012): forward, addr) admin(10012): File "/usr/local/mailman/Mailman/ListAdmin.py", line 353, in __handlepost admin(10012): syslog('vette', note) admin(10012): File "/usr/local/mailman/Mailman/Logging/Syslog.py", line 40, in write admin(10012): self.write_ex(kind, msg, args, kws) admin(10012): File "/usr/local/mailman/Mailman/Logging/Syslog.py", line 58, in write_ex admin(10012): logf.write(msg + '\n') admin(10012): File "/usr/local/mailman/Mailman/Logging/StampedLogger.py", line 73, in write admin(10012): Logger.write(self, "%s %s" % (prefix, msg)) admin(10012): File "/usr/local/mailman/Mailman/Logging/Logger.py", line 91, in write admin(10012): f.write(msg) admin(10012): UnicodeEncodeError: 'ascii' codec can't encode character u'\xf3' in position 99: ordinal not in range(128) admin(10012): [----- Python Information -----] admin(10012): sys.version = 2.4.2 (#2, Nov 7 2005, 19:05:41) [GCC 2.95.4 20020320 [FreeBSD]] admin(10012): sys.executable = /usr/local/bin/python admin(10012): sys.prefix = /usr/local admin(10012): sys.exec_prefix = /usr/local admin(10012): sys.path = /usr/local admin(10012): sys.platform = freebsd4 admin(10012): [----- Environment Variables -----] admin(10012): HTTP_REFERER: http://list.uncanny.net/mailman/admindb/wsa-talk admin(10012): SERVER_SOFTWARE: Apache/1.3.33 admin(10012): SCRIPT_NAME: /mailman/admindb admin(10012): SERVER_SIGNATURE: <ADDRESS>Apache/1.3.33 Server at list.uncanny.net Port 80</ADDRESS> admin(10012): admin(10012): REQUEST_METHOD: POST admin(10012): HTTP_KEEP_ALIVE: 300 admin(10012): SERVER_PROTOCOL: HTTP/1.1 admin(10012): QUERY_STRING: admin(10012): CONTENT_LENGTH: 61108 admin(10012): HTTP_ACCEPT_CHARSET: ISO-8859-1, utf-8;q=0.66, *;q=0.66 admin(10012): HTTP_USER_AGENT: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.2.1) Gecko/20010901 admin(10012): HTTP_CONNECTION: keep-alive admin(10012): HTTP_COOKIE: admin(10012): SERVER_NAME: list.uncanny.net admin(10012): REMOTE_ADDR: 192.168.49.49 admin(10012): PATH_TRANSLATED: /Web/hosts/list.uncanny.net/public_html/wsa-talk admin(10012): SERVER_PORT: 80 admin(10012): SERVER_ADDR: 192.168.100.225 admin(10012): DOCUMENT_ROOT: /Web/hosts/list.uncanny.net/public_html admin(10012): PYTHONPATH: /usr/local/mailman admin(10012): SCRIPT_FILENAME: /Web/hosts/list.uncanny.net/public_html/mailman/admindb admin(10012): SERVER_ADMIN: webmaster@uncanny.net admin(10012): HTTP_HOST: list.uncanny.net admin(10012): REQUEST_URI: /mailman/admindb/wsa-talk admin(10012): HTTP_ACCEPT: text/xml, application/xml, application/xhtml+xml, text/html;q=0.9, image/png, image/jpeg, image/gif;q=0.2, text/plain;q=0.8, text/css, */*;q=0.1 admin(10012): GATEWAY_INTERFACE: CGI/1.1 admin(10012): REMOTE_PORT: 33142 admin(10012): HTTP_ACCEPT_LANGUAGE: en-us admin(10012): CONTENT_TYPE: application/x-www-form-urlencoded admin(10012): HTTP_ACCEPT_ENCODING: gzip,deflate,compress,identity admin(10012): UNIQUE_ID: Q5yWc8CoZOEAACbkeKA admin(10012): PATH_INFO: /wsa-talk ========== TRACEBACK END -- Edward Elhauge <ee@uncanny.net> "One allows himself to be fooled once, to be fooled twice; but he who permits himself always to be fooled remains a fool." -- Rudolf Rocker
Hi, Can you test attached patch in your system? My system accepts non-ascii characters in logging and does not reproduce your error. Edward Elhauge wrote:
I asked the Mailman-Users list this question last week, but got no useful replies. Since this is a bug in the code, someone here might know how to proceed.
I might add to the original question: Would it help to go to the very latest Mailman release?
RRRRRRRRRRRRRRRRRRRRRRRR
I'd like some help on dealing with this problem permanently. I've been getting this behavior every couple of months and haven't found a good solution yet.
The way it happens is that I'm on the "Administrative requests" page for some mailing list and when I submit the deletion requests I get the following reply from the browser: Bug in Mailman version 2.1.6
We're sorry, we hit a bug!
Please inform the webmaster for this site of this problem. Printing of traceback and other system information has been explicitly inhibited, but the webmaster can find this information in the Mailman error logs.
Usually you can then go back an delete individual entries, but there will be one or more that produce the error, with this sort of traceback (below):
I take it that the email in question (SPAM email) isn't properly encoded. Any idea on how to get around this problem? In the past I've gone into the queues and deleted files, etc, but the process isn't easy, and of course I won't get any cooperation from the SPAMers sending the messages.
The OS is FreeBSD 4.9 and Mailman is 2.1.6_1, built from source.
========== TRACEBACK START Dec 11 13:13:52 2005 admin(10012): @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ admin(10012): [----- Mailman Version: 2.1.6 -----] admin(10012): [----- Traceback ------] admin(10012): Traceback (most recent call last): admin(10012): File "/usr/local/mailman/scripts/driver", line 101, in run_main admin(10012): main() admin(10012): File "/usr/local/mailman/Mailman/Cgi/admindb.py", line 163, in main admin(10012): process_form(mlist, doc, cgidata) admin(10012): File "/usr/local/mailman/Mailman/Cgi/admindb.py", line 713, in process_form admin(10012): forward, forwardaddr) admin(10012): File "/usr/local/mailman/Mailman/ListAdmin.py", line 167, in HandleRequest admin(10012): forward, addr) admin(10012): File "/usr/local/mailman/Mailman/ListAdmin.py", line 353, in __handlepost admin(10012): syslog('vette', note) admin(10012): File "/usr/local/mailman/Mailman/Logging/Syslog.py", line 40, in write admin(10012): self.write_ex(kind, msg, args, kws) admin(10012): File "/usr/local/mailman/Mailman/Logging/Syslog.py", line 58, in write_ex admin(10012): logf.write(msg + '\n') admin(10012): File "/usr/local/mailman/Mailman/Logging/StampedLogger.py", line 73, in write admin(10012): Logger.write(self, "%s %s" % (prefix, msg)) admin(10012): File "/usr/local/mailman/Mailman/Logging/Logger.py", line 91, in write admin(10012): f.write(msg) admin(10012): UnicodeEncodeError: 'ascii' codec can't encode character u'\xf3' in position 99: ordinal not in range(128) admin(10012): [----- Python Information -----] admin(10012): sys.version = 2.4.2 (#2, Nov 7 2005, 19:05:41) [GCC 2.95.4 20020320 [FreeBSD]] admin(10012): sys.executable = /usr/local/bin/python admin(10012): sys.prefix = /usr/local admin(10012): sys.exec_prefix = /usr/local admin(10012): sys.path = /usr/local admin(10012): sys.platform = freebsd4 admin(10012): [----- Environment Variables -----] admin(10012): HTTP_REFERER: http://list.uncanny.net/mailman/admindb/wsa-talk admin(10012): SERVER_SOFTWARE: Apache/1.3.33 admin(10012): SCRIPT_NAME: /mailman/admindb admin(10012): SERVER_SIGNATURE: <ADDRESS>Apache/1.3.33 Server at list.uncanny.net Port 80</ADDRESS> admin(10012): admin(10012): REQUEST_METHOD: POST admin(10012): HTTP_KEEP_ALIVE: 300 admin(10012): SERVER_PROTOCOL: HTTP/1.1 admin(10012): QUERY_STRING: admin(10012): CONTENT_LENGTH: 61108 admin(10012): HTTP_ACCEPT_CHARSET: ISO-8859-1, utf-8;q=0.66, *;q=0.66 admin(10012): HTTP_USER_AGENT: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.2.1) Gecko/20010901 admin(10012): HTTP_CONNECTION: keep-alive admin(10012): HTTP_COOKIE: admin(10012): SERVER_NAME: list.uncanny.net admin(10012): REMOTE_ADDR: 192.168.49.49 admin(10012): PATH_TRANSLATED: /Web/hosts/list.uncanny.net/public_html/wsa-talk admin(10012): SERVER_PORT: 80 admin(10012): SERVER_ADDR: 192.168.100.225 admin(10012): DOCUMENT_ROOT: /Web/hosts/list.uncanny.net/public_html admin(10012): PYTHONPATH: /usr/local/mailman admin(10012): SCRIPT_FILENAME: /Web/hosts/list.uncanny.net/public_html/mailman/admindb admin(10012): SERVER_ADMIN: webmaster@uncanny.net admin(10012): HTTP_HOST: list.uncanny.net admin(10012): REQUEST_URI: /mailman/admindb/wsa-talk admin(10012): HTTP_ACCEPT: text/xml, application/xml, application/xhtml+xml, text/html;q=0.9, image/png, image/jpeg, image/gif;q=0.2, text/plain;q=0.8, text/css, */*;q=0.1 admin(10012): GATEWAY_INTERFACE: CGI/1.1 admin(10012): REMOTE_PORT: 33142 admin(10012): HTTP_ACCEPT_LANGUAGE: en-us admin(10012): CONTENT_TYPE: application/x-www-form-urlencoded admin(10012): HTTP_ACCEPT_ENCODING: gzip,deflate,compress,identity admin(10012): UNIQUE_ID: Q5yWc8CoZOEAACbkeKA admin(10012): PATH_INFO: /wsa-talk ========== TRACEBACK END
-- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/ Index: Syslog.py =================================================================== RCS file: /cvsroot/mailman/mailman/Mailman/Logging/Syslog.py,v retrieving revision 2.3.2.1 diff -u -r2.3.2.1 Syslog.py --- Syslog.py 27 Aug 2005 01:40:16 -0000 2.3.2.1 +++ Syslog.py 15 Dec 2005 02:42:57 -0000 @@ -55,7 +55,12 @@ # It's really bad if exceptions in the syslogger cause other crashes except Exception, e: msg = 'Bad format "%s": %s: %s' % (origmsg, repr(e), e) - logf.write(msg + '\n') + try: + logf.write(msg + '\n') + except UnicodeError: + # Python 2.4 may fail to write 8bit (non-ascii) characters + import quopri + logf.write(quopri.encodestring(msg) + '\n') # For the ultimate in convenience __call__ = write
- Tokio Kikuchi <tkikuchi@is.kochi-u.ac.jp> wrote on [2005-12-14 18:49]:
Hi,
Can you test attached patch in your system? My system accepts non-ascii characters in logging and does not reproduce your error.
Your patch worked. And yes I have Python 2.4.2. Thank you very much.
I'd like to ask this list another question. Would there be any philosophical objection to a config flag in Python that would /dev/null the "Post by non-member to a members-only list"?
I've done this in the past, for older versions, but not in a configurable form. I'd be would be willing to submit a patch for this as a per-maillist configurable feature if it had a chance to be folded into a release.
This feature makes sense if you have a heavily spam targeted list. It's also less blunt than the "Discard all messages marked Defer".
Edward Elhauge wrote:
I asked the Mailman-Users list this question last week, but got no useful replies. Since this is a bug in the code, someone here might know how to proceed.
I might add to the original question: Would it help to go to the very latest Mailman release?
RRRRRRRRRRRRRRRRRRRRRRRR
I'd like some help on dealing with this problem permanently. I've been getting this behavior every couple of months and haven't found a good solution yet.
The way it happens is that I'm on the "Administrative requests" page for some mailing list and when I submit the deletion requests I get the following reply from the browser: Bug in Mailman version 2.1.6
...
-- Edward Elhauge <ee@uncanny.net> "One allows himself to be fooled once, to be fooled twice; but he who permits himself always to be fooled remains a fool." -- Rudolf Rocker
On Dec 15, 2005, at 03:39, Edward Elhauge wrote:
I'd like to ask this list another question. Would there be any philosophical objection to a config flag in Python that would /dev/null the "Post by non-member to a members-only list"?
I've done this in the past, for older versions, but not in a configurable form. I'd be would be willing to submit a patch for this as a per-maillist configurable feature if it had a chance to be folded into a release.
Then I take it you haven't used a "newer" version yet? It sounds like you want to set DEFAULT_GENERIC_NONMEMBER_ACTION = 3 in mm_cfg.py, or twiddle the setting to "Discard" on some of your lists. The generic_nonmember_setting appears second from the bottom on the admin/<listname>/privacy/sender page.
--Robby
participants (3)
-
Edward Elhauge -
Robby Griffin -
Tokio Kikuchi