Mailman 3 April 2005 - Mailman-coders

[ mailman-Bugs-1168999 ] OutgoingRunner gets in expensive recursive loop
by SourceForge.net April 26, 2005

April 26, 2005

Bugs item #1168999, was opened at 2005-03-23 18:30 Message generated for change (Comment added) made by kjd You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1168999&group_… Category: mail delivery Group: 2.1 beta Status: Open Resolution: None Priority: 5 Submitted By: Kim Davies (kjd) Assigned to: Nobody/Anonymous (nobody) Summary: OutgoingRunner gets in expensive recursive loop Initial Comment: I have had a problem spring up the past few weeks where the OutgoingRunner gets in a loop which effectively brings down the machine by spiking the CPU to 99%. Running "strace" on the process I see it constantly deleting and reimplanting the same queue file in qfiles/out/ over and over, many times per second. The initial problem inurred with a 2.1.2 install, but installing 2.1.6b4 shows the same. Unfortunately the problem is somewhat ephemeral when trying to diagnose it - if I manage to kill the OutgoingRunner between a read and write, the queue file gets lost the the problem disappears for a while. I don't know if it is useful, but attached is the strace output of a complete read/write cycle. I haven't had the opportunity to further debug it (by stepping through the python) as currently I am not in this state. I am not sure how long it will be until it is triggered again, but it has happened about 4 times in the past two weeks. It has never occured before this over 3 years. I consider this issue fairly problematic - the machine becomes unusable when it reaches this state due to CPU exhaustion. Any tips of helping isolate the problem are welcome. I have modified mailmanctl to run all queuerunners with a verbose flag, so next time maybe there will be useful information logged. ---------------------------------------------------------------------- >Comment By: Kim Davies (kjd) Date: 2005-04-26 15:30 Message: Logged In: YES user_id=168657 I have managed to captures a number of qfiles that are causing this phenomenon (which is recurring more often the past few weeks). They have the following properties: - They are all "Post by non-member to a members-only list " responses to spam that has gone to a moderated list. - They come from non-existant domains. Here is a sample dump of one of the .db files from the queue that is looping: $ /usr/local/mailman/bin/dumpdb 1114500259.9807329+a1518ee474d8edb0e83615df60270885165d5f83.db { 'deliver_after': 1114503867.0899999, 'deliver_until': 1114932267.0899999, 'lang': 'en', 'last_recip_count': 1, 'listname': 'ga', 'nodecorate': 1, 'original_sender': 'ga-bounces(a)lists.centr.org', 'personalize': 1, 'pipeline': [], 'received_time': 1114500259.9807329, 'recips': ['sidfks(a)sklfislxkd.com'], 'reduced_list_headers': 1, 'verp': 1, 'version': 3} ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1168999&group_…

1 0

[ mailman-Patches-1189243 ] Unhandle exception in Message.get_content_charset()
by SourceForge.net April 25, 2005

April 25, 2005

Patches item #1189243, was opened at 2005-04-25 00:16 Message generated for change (Comment added) made by bwarsaw You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1189243&group_… Category: None Group: Mailman 2.1 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Jon Parise (jparise) Assigned to: Nobody/Anonymous (nobody) Summary: Unhandle exception in Message.get_content_charset() Initial Comment: Message.get_content_charset() can encounter an unhandled exception "(TypeError: unicode() argument 2 must be string, not None") when charset is None. This can can only occur when the call to email.Message.Message.get_content_charset(self, failobj) itself returns None, but, not being acquainted with the email package, I don't understand the circumstances which can cause this situation. Regardless, when charset is None, the call to unicode() will throw the TypeError exception. The attached patch recognizes this as a valid failure case such that failobj will be returned. ---------------------------------------------------------------------- >Comment By: Barry A. Warsaw (bwarsaw) Date: 2005-04-25 13:02 Message: Logged In: YES user_id=12800 If it's not too much trouble. I'd love to have a test case in the email package for the TypeError exception. If you can't, no worries. ---------------------------------------------------------------------- Comment By: Jon Parise (jparise) Date: 2005-04-25 11:46 Message: Logged In: YES user_id=485579 Sorry, I don't. Those messages were unshunted as a result of my fix. I could try to instrument a collection mechanism by trapping that exception and storing a copy of the offensive message, however. Would that still be useful? ---------------------------------------------------------------------- Comment By: Barry A. Warsaw (bwarsaw) Date: 2005-04-25 10:32 Message: Logged In: YES user_id=12800 Is there any possibility that you have a sample message that exhibits this problem? I would /really/ like to add a test case for this to the email package, which is where the patch really belongs. ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2005-04-25 00:50 Message: Logged In: YES user_id=67709 Thank you! I just fixed in CVS. (I'd rather make it bare except ...) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1189243&group_…

1 0

[ mailman-Patches-1189243 ] Unhandle exception in Message.get_content_charset()
by SourceForge.net April 25, 2005

April 25, 2005

Patches item #1189243, was opened at 2005-04-24 21:16 Message generated for change (Comment added) made by jparise You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1189243&group_… Category: None Group: Mailman 2.1 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Jon Parise (jparise) Assigned to: Nobody/Anonymous (nobody) Summary: Unhandle exception in Message.get_content_charset() Initial Comment: Message.get_content_charset() can encounter an unhandled exception "(TypeError: unicode() argument 2 must be string, not None") when charset is None. This can can only occur when the call to email.Message.Message.get_content_charset(self, failobj) itself returns None, but, not being acquainted with the email package, I don't understand the circumstances which can cause this situation. Regardless, when charset is None, the call to unicode() will throw the TypeError exception. The attached patch recognizes this as a valid failure case such that failobj will be returned. ---------------------------------------------------------------------- >Comment By: Jon Parise (jparise) Date: 2005-04-25 08:46 Message: Logged In: YES user_id=485579 Sorry, I don't. Those messages were unshunted as a result of my fix. I could try to instrument a collection mechanism by trapping that exception and storing a copy of the offensive message, however. Would that still be useful? ---------------------------------------------------------------------- Comment By: Barry A. Warsaw (bwarsaw) Date: 2005-04-25 07:32 Message: Logged In: YES user_id=12800 Is there any possibility that you have a sample message that exhibits this problem? I would /really/ like to add a test case for this to the email package, which is where the patch really belongs. ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2005-04-24 21:50 Message: Logged In: YES user_id=67709 Thank you! I just fixed in CVS. (I'd rather make it bare except ...) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1189243&group_…

1 0

[ mailman-Patches-1189243 ] Unhandle exception in Message.get_content_charset()
by SourceForge.net April 25, 2005

April 25, 2005

Patches item #1189243, was opened at 2005-04-25 00:16 Message generated for change (Comment added) made by bwarsaw You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1189243&group_… Category: None Group: Mailman 2.1 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Jon Parise (jparise) Assigned to: Nobody/Anonymous (nobody) Summary: Unhandle exception in Message.get_content_charset() Initial Comment: Message.get_content_charset() can encounter an unhandled exception "(TypeError: unicode() argument 2 must be string, not None") when charset is None. This can can only occur when the call to email.Message.Message.get_content_charset(self, failobj) itself returns None, but, not being acquainted with the email package, I don't understand the circumstances which can cause this situation. Regardless, when charset is None, the call to unicode() will throw the TypeError exception. The attached patch recognizes this as a valid failure case such that failobj will be returned. ---------------------------------------------------------------------- >Comment By: Barry A. Warsaw (bwarsaw) Date: 2005-04-25 10:32 Message: Logged In: YES user_id=12800 Is there any possibility that you have a sample message that exhibits this problem? I would /really/ like to add a test case for this to the email package, which is where the patch really belongs. ---------------------------------------------------------------------- Comment By: Tokio Kikuchi (tkikuchi) Date: 2005-04-25 00:50 Message: Logged In: YES user_id=67709 Thank you! I just fixed in CVS. (I'd rather make it bare except ...) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1189243&group_…

1 0

[ mailman-Patches-1189243 ] Unhandle exception in Message.get_content_charset()
by SourceForge.net April 25, 2005

April 25, 2005

Patches item #1189243, was opened at 2005-04-25 04:16 Message generated for change (Comment added) made by tkikuchi You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1189243&group_… Category: None Group: Mailman 2.1 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Jon Parise (jparise) Assigned to: Nobody/Anonymous (nobody) Summary: Unhandle exception in Message.get_content_charset() Initial Comment: Message.get_content_charset() can encounter an unhandled exception "(TypeError: unicode() argument 2 must be string, not None") when charset is None. This can can only occur when the call to email.Message.Message.get_content_charset(self, failobj) itself returns None, but, not being acquainted with the email package, I don't understand the circumstances which can cause this situation. Regardless, when charset is None, the call to unicode() will throw the TypeError exception. The attached patch recognizes this as a valid failure case such that failobj will be returned. ---------------------------------------------------------------------- >Comment By: Tokio Kikuchi (tkikuchi) Date: 2005-04-25 04:50 Message: Logged In: YES user_id=67709 Thank you! I just fixed in CVS. (I'd rather make it bare except ...) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1189243&group_…

1 0

[ mailman-Patches-1189243 ] Unhandle exception in Message.get_content_charset()
by SourceForge.net April 25, 2005

April 25, 2005

Patches item #1189243, was opened at 2005-04-24 21:16 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1189243&group_… Category: None Group: Mailman 2.1 Status: Open Resolution: None Priority: 5 Submitted By: Jon Parise (jparise) Assigned to: Nobody/Anonymous (nobody) Summary: Unhandle exception in Message.get_content_charset() Initial Comment: Message.get_content_charset() can encounter an unhandled exception "(TypeError: unicode() argument 2 must be string, not None") when charset is None. This can can only occur when the call to email.Message.Message.get_content_charset(self, failobj) itself returns None, but, not being acquainted with the email package, I don't understand the circumstances which can cause this situation. Regardless, when charset is None, the call to unicode() will throw the TypeError exception. The attached patch recognizes this as a valid failure case such that failobj will be returned. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1189243&group_…

1 0

[ mailman-Patches-1160353 ] XHTML Compliant Web UI
by SourceForge.net April 24, 2005

April 24, 2005

Patches item #1160353, was opened at 2005-03-09 21:40 Message generated for change (Comment added) made by carbonnb You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1160353&group_… Category: Web UI Group: Mailman 2.1 Status: Open Resolution: None Priority: 5 Submitted By: Bryan Carbonnell (carbonnb) Assigned to: Nobody/Anonymous (nobody) Summary: XHTML Compliant Web UI Initial Comment: This is a patch that will make the Web UI XHTML 1.0 Strict compliant. Currently I only have the listinfo pages XHTML compliant, but as time goes by I'm going to make the entire UI compliant. This also allows some small amount of CSS formatting as well. Hopefully this will be of use to somebody. ---------------------------------------------------------------------- >Comment By: Bryan Carbonnell (carbonnb) Date: 2005-04-24 08:21 Message: Logged In: YES user_id=449953 The patch for MM 2.1.6-RC1 and above makes the entire WEB UI XHTML1 compliant, except for the archive pages, since they are HTM 3.2 compliant already. To use this patch, unzip it, switch to your source directory and issue the command patch -p1 <path/to/xhtml-2.1.6rc1-0.1.patch ---------------------------------------------------------------------- Comment By: Bryan Carbonnell (carbonnb) Date: 2005-03-09 21:47 Message: Logged In: YES user_id=449953 To use this patch, unzip it, switch to your source directory and issue the command patch -p1 <path/to/xhtml-2.1.6b4-0.1.patch ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1160353&group_…

1 0

[ mailman-Bugs-1188133 ] CGI group id not properly tested
by SourceForge.net April 23, 2005

April 23, 2005

Bugs item #1188133, was opened at 2005-04-22 14:58 Message generated for change (Comment added) made by tkikuchi You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1188133&group_… Category: Web/CGI Group: 2.1 (stable) >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Graham Klyne (grahamk) Assigned to: Nobody/Anonymous (nobody) Summary: CGI group id not properly tested Initial Comment: [I tried to send this to mailman-developers, but my message was discarded] I've just downloaded and installed the latest mailman 2.1.6rc1 and encountered a CGI permissions problem (running with Apache 2.0 on Scientific Linux 3.04), for which a patch is described in: http://minaret.biz/tips/mailman.html (briefly, replace getgid with getegid in common.c) Applying this patch resolves the problem I was experiencing. Is there any reason this isn't applied in the mailman distribution? #g ---------------------------------------------------------------------- >Comment By: Tokio Kikuchi (tkikuchi) Date: 2005-04-23 23:49 Message: Logged In: YES user_id=67709 The 'patch' and discussion in the page was invalid and updated by the author. In general, you should not patch the wrapper program. You can also read a good article on the mailman security mechanism by John Dennis here: http://mail.python.org/pipermail/mailman-developers/2005-April/017996.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1188133&group_…

1 0

[ mailman-Patches-1188240 ] correct links, if webserver is forwarded
by SourceForge.net April 22, 2005

April 22, 2005

Patches item #1188240, was opened at 2005-04-22 18:32 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1188240&group_… Category: Web UI Group: Mailman 2.1 Status: Open Resolution: None Priority: 5 Submitted By: pille (zaarkov) Assigned to: Nobody/Anonymous (nobody) Summary: correct links, if webserver is forwarded Initial Comment: in my setup there are two servers. one ist the mail-server, the other one the webserver. the mail-server uses an own apache on itself to display the web UI, but it is only accessible throu the webserver that forwards connections to it (using mod_proxy's ProxyPass-directive). the links that are generated on sites like http://.../listinfo that don't know anything about virtual hosts refer to the HTTP_HOST environment variable, which represents the mail-server and not the webserver. therefore this patch tries to evaluate the HTTP_X_FORWARDED_HOST and use it if it exists. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1188240&group_…

1 0

[ mailman-Patches-1188237 ] build correct links, if webserver is forwarded
by SourceForge.net April 22, 2005

April 22, 2005

Patches item #1188237, was opened at 2005-04-22 18:26 Message generated for change (Settings changed) made by zaarkov You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1188237&group_… Category: Web UI Group: Mailman 2.1 >Status: Deleted Resolution: None Priority: 5 Submitted By: pille (zaarkov) Assigned to: Nobody/Anonymous (nobody) Summary: build correct links, if webserver is forwarded Initial Comment: in my setup there are two servers. one ist the mail-server, the other one the webserver. the mail-server uses an own apache on itself to display the web UI, but it is only accessible throu the webserver that forwards connections to it (using mod_proxy's ProxyPass-directive). the links that are generated on sites like http://.../listinfo that don't know anything about virtual hosts refer to the HTTP_HOST environment variable, which represents the mail-server and not the webserver. therefore this patch tries to evaluate the HTTP_X_FORWARDED_HOST and use it if it exists. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1188237&group_…

1 0