Bugs item #1908112, was opened at 2008-03-05 17:02 Message generated for change (Comment added) made by shagedorn You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1908112&group_id=103 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: 2.1 (stable) Status: Open Resolution: None Priority: 5 Private: No Submitted By: Sebastian Hagedorn (shagedorn) Assigned to: Mark Sapiro (msapiro) Summary: Uncaught runner exception: iteration over non-sequence Initial Comment: I don't normally check /var/log/mailman/error as long as everything seems to be working, but today I noticed loads of errors like this one: Mar 04 07:50:44 2008 (4711) SHUNTING: 1167220351.925714+8af35bc9b49ca21d542d7fac99bdb87a688619ef Mar 04 07:50:44 2008 (4711) Uncaught runner exception: iteration over non-sequence Mar 04 07:50:44 2008 (4711) Traceback (most recent call last): File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 112, in _oneloop self._onefile(msg, msgdata) File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 170, in _onefile keepqueued = self._dispose(mlist, msg, msgdata) File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 130, in _dispose more = self._dopipeline(mlist, msg, msgdata, pipeline) File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 153, in _dopipeline sys.modules[modname].process(mlist, msg, msgdata) File "/usr/lib/mailman/Mailman/Handlers/SpamDetect.py", line 111, in process g.flatten(p) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 101, in flatten self._write(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 129, in _write self._dispatch(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 155, in _dispatch meth(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 226, in _handle_multipart g.flatten(part, unixfrom=False) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 101, in flatten self._write(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 129, in _write self._dispatch(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 155, in _dispatch meth(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 293, in _handle_message g.flatten(msg.get_payload(0), unixfrom=False) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 101, in flatten self._write(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 129, in _write self._dispatch(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 155, in _dispatch meth(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 226, in _handle_multipart g.flatten(part, unixfrom=False) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 101, in flatten self._write(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 129, in _write self._dispatch(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 155, in _dispatch meth(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 273, in _handle_message_delivery_status g.flatten(part, unixfrom=False) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 101, in flatten self._write(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 129, in _write self._dispatch(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 155, in _dispatch meth(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 270, in _handle_message_delivery_status for part in msg.get_payload(): TypeError: iteration over non-sequence I can provide more samples if necessary. I have searched the tracker for other reports like this, but haven't seen any ... ----------------------------------------------------------------------

Comment By: Sebastian Hagedorn (shagedorn) Date: 2008-03-06 09:31

Message: Logged In: YES user_id=309132 Originator: YES Hi, thanks for looking into this. To address your questions: - we are running 2.1.9 - we have a cronjob in cron.daily that does an "unshunt" - the file I mentioned originally doesn't exist anymore (because it has been unshunted?) - the file I attached had this error message: Mar 05 07:56:19 2008 (4711) SHUNTING: 1167220351.925714+67431f92e260e4da420214c9a343ace677ca1acd Mar 05 07:56:19 2008 (4711) Uncaught runner exception: iteration over non-sequence Mar 05 07:56:19 2008 (4711) Traceback (most recent call last): File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 112, in _oneloop self._onefile(msg, msgdata) File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 170, in _onefile keepqueued = self._dispose(mlist, msg, msgdata) ... So the error is different from what I see with show_qfiles. - We have not modified the charset of German to UTF-8 Now my question is whether the "unshunt" cron job is evil? I confess I never properly understood how it works. It seemed to get messages delivered that were shunted originally, so at some point I set up that cronjob ... I suppose I should just remove all files that are too old. But: how do I see the contents when show_qfiles does not work? Are there other tools? Thanks for all your help! ---------------------------------------------------------------------- Comment By: Mark Sapiro (msapiro) Date: 2008-03-06 06:55 Message: Logged In: YES user_id=1123998 Originator: NO I have looked further at the shunt queue entry 1167220351.925714+67431f92e260e4da420214c9a343ace677ca1acd.pck. Here's what I see. This message was a password reminder, originally generated by mailman on May 1, 2006. It was addressed to a user whose preferred language appears to be Korean as it is 'Content-Type: text/plain, charset="euc-kr"' with 'Subject: =?euc-kr?q?uni-koeln=2Ede_mailing_list_memberships_reminder?='. Apparently there was some error at that time which caused the message to be shunted. I am a bit confused about what happened next because the time stamp on the queue entry and the received_time in the message's metadata are 1167220351.925714 which is 11:52:31 Dec 27, 2006 +0000. However, the message metadata says the message was originally in the 'in' queue, so I think it must have been unshunted to the 'in' queue where it couldn't be properly processed, causing it to be shunted again in December, 2006. The question now is what is causing these messages to be reprocessed at this point, and why would a message that threw an exception in trying to flatten the message in SpamDetect be shunted with a 14+ month old time stamp? In any case, I think the current error with this particular message is due to it's being unshunted to the wrong queue. ---------------------------------------------------------------------- Comment By: Mark Sapiro (msapiro) Date: 2008-03-05 17:58 Message: Logged In: YES user_id=1123998 Originator: NO The file you attached appears to be a password reminder with a lot of garbled unicode. Have you changed the characterset for the german language to utf-8? There is some issue with the templates/de/cronpass.txt template or a list specific version of it. ---------------------------------------------------------------------- Comment By: Mark Sapiro (msapiro) Date: 2008-03-05 17:47 Message: Logged In: YES user_id=1123998 Originator: NO The queue entry you attached is not the one from the reported traceback and it seems to have a different problem. When I attempt to look at it with bin/dumpdb or bin/show_qfiles, I get UnicodeEncodeError: 'ascii' codec can't encode character u'\uc774' in position 0: ordinal not in range(128) in a different part of Generator.py. I see that you see the same. What is your Mailman version? Also, if I look at the contents of the queue entry, it appears to be a password reminder which is strange. I don't have time to look at this further right now, but if you can, can you find the 1167220351.925714+8af35bc9b49ca21d542d7fac99bdb87a688619ef.pck and attach that. Also, is the error in Mailman's error log for the 1167220351.925714+67431f92e260e4da420214c9a343ace677ca1acd.pck file the same UnicodeEncodeError that you get from show_qfiles and dumpdb, or is it the TypeError as in the original report? ---------------------------------------------------------------------- Comment By: Sebastian Hagedorn (shagedorn) Date: 2008-03-05 17:22 Message: Logged In: YES user_id=309132 Originator: YES OK, I have attached the file. Its name is different, but I guess it's the same(?): # grep 1167220351.925714 /var/log/mailman/error Mar 02 07:37:24 2008 (4711) SHUNTING: 1167220351.925714+55bb5ab698f528a8061d4ef867a634abbed6769e Mar 03 07:36:03 2008 (4711) SHUNTING: 1167220351.925714+d29bf7b17b742fca8b0b71c09c1e52ebcf08620e Mar 04 07:50:44 2008 (4711) SHUNTING: 1167220351.925714+8af35bc9b49ca21d542d7fac99bdb87a688619ef Mar 05 07:56:19 2008 (4711) SHUNTING: 1167220351.925714+67431f92e260e4da420214c9a343ace677ca1acd Neither show_qfiles nor dbdump works: /usr/lib/mailman/bin/show_qfiles /var/spool/mailman/shunt/1167220351.925714+67431f92e260e4da420214c9a343ace677ca1acd.pck ====================> /var/spool/mailman/shunt/1167220351.925714+67431f92e260e4da420214c9a343ace677ca1acd.pck Traceback (most recent call last): File "/usr/lib/mailman/bin/show_qfiles", line 95, in ? main() File "/usr/lib/mailman/bin/show_qfiles", line 88, in main sys.stdout.write(msg.as_string()) File "/usr/lib/mailman/pythonlib/email/Message.py", line 135, in as_string g.flatten(self, unixfrom=unixfrom) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 101, in flatten self._write(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 129, in _write self._dispatch(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 155, in _dispatch meth(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 198, in _handle_text self._fp.write(payload) UnicodeError: ASCII encoding error: ordinal not in range(128) [root@lvr13 Mailman]# /usr/lib/mailman/bin/dumpdb /var/spool/mailman/shunt/1167220351.925714+67431f92e260e4da420214c9a343ace677ca1acd.pck [----- start pickle file -----] <----- start object 1 -----> Traceback (most recent call last): File "/usr/lib/mailman/bin/dumpdb", line 159, in ? msg = main() File "/usr/lib/mailman/bin/dumpdb", line 149, in main pp.pprint(obj) File "/usr/lib/python2.2/pprint.py", line 110, in pprint self.__stream.write(self.pformat(object) + "\n") File "/usr/lib/python2.2/pprint.py", line 114, in pformat self.__format(object, sio, 0, 0, {}, 0) File "/usr/lib/python2.2/pprint.py", line 136, in __format rep = self.__repr(object, context, level - 1) File "/usr/lib/python2.2/pprint.py", line 200, in __repr self.__depth, level) File "/usr/lib/python2.2/pprint.py", line 287, in _safe_repr rep = `object` File "/usr/lib/mailman/Mailman/Message.py", line 51, in __repr__ return self.__str__() File "/usr/lib/mailman/pythonlib/email/Message.py", line 121, in __str__ return self.as_string(unixfrom=True) File "/usr/lib/mailman/pythonlib/email/Message.py", line 135, in as_string g.flatten(self, unixfrom=unixfrom) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 101, in flatten self._write(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 129, in _write self._dispatch(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 155, in _dispatch meth(msg) File "/usr/lib/mailman/pythonlib/email/Generator.py", line 198, in _handle_text self._fp.write(payload) UnicodeError: ASCII encoding error: ordinal not in range(128) ---------------------------------------------------------------------- Comment By: Sebastian Hagedorn (shagedorn) Date: 2008-03-05 17:19 Message: Logged In: YES user_id=309132 Originator: YES File Added: 1167220351.925714+67431f92e260e4da420214c9a343ace677ca1acd.pck ---------------------------------------------------------------------- Comment By: Mark Sapiro (msapiro) Date: 2008-03-05 17:14 Message: Logged In: YES user_id=1123998 Originator: NO There is a problem with the message. Try running bin/show_qfiles or if that doesn't work, bin/dumpdb on one or more of the shunted files (e.g. qfiles/shunt/1167220351.925714+8af35bc9b49ca21d542d7fac99bdb87a688619ef.pck). In any case, can you send me this file or attach it to this report. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1908112&group_id=103