[ mailman-Bugs-1908112 ] Uncaught runner exception: iteration over non-sequence

SourceForge.net noreply at sourceforge.net
Thu Mar 6 09:31:36 CET 2008


Bugs item #1908112, was opened at 2008-03-05 17:02
Message generated for change (Comment added) made by shagedorn
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1908112&group_id=103

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: 2.1 (stable)
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Sebastian Hagedorn (shagedorn)
Assigned to: Mark Sapiro (msapiro)
Summary: Uncaught runner exception: iteration over non-sequence

Initial Comment:
I don't normally check /var/log/mailman/error as long as everything seems to be working, but today I noticed loads of errors like this one:

Mar 04 07:50:44 2008 (4711) SHUNTING: 1167220351.925714+8af35bc9b49ca21d542d7fac99bdb87a688619ef
Mar 04 07:50:44 2008 (4711) Uncaught runner exception: iteration over non-sequence
Mar 04 07:50:44 2008 (4711) Traceback (most recent call last):
  File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 112, in _oneloop
    self._onefile(msg, msgdata)
  File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 170, in _onefile
    keepqueued = self._dispose(mlist, msg, msgdata)
  File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 130, in _dispose
    more = self._dopipeline(mlist, msg, msgdata, pipeline)
  File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 153, in _dopipeline
    sys.modules[modname].process(mlist, msg, msgdata)
  File "/usr/lib/mailman/Mailman/Handlers/SpamDetect.py", line 111, in process
    g.flatten(p)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 101, in flatten
    self._write(msg)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 129, in _write
    self._dispatch(msg)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 155, in _dispatch
    meth(msg)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 226, in _handle_multipart
    g.flatten(part, unixfrom=False)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 101, in flatten
    self._write(msg)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 129, in _write
    self._dispatch(msg)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 155, in _dispatch
    meth(msg)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 293, in _handle_message
    g.flatten(msg.get_payload(0), unixfrom=False)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 101, in flatten
    self._write(msg)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 129, in _write
    self._dispatch(msg)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 155, in _dispatch
    meth(msg)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 226, in _handle_multipart
    g.flatten(part, unixfrom=False)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 101, in flatten
    self._write(msg)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 129, in _write
    self._dispatch(msg)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 155, in _dispatch
    meth(msg)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 273, in _handle_message_delivery_status
    g.flatten(part, unixfrom=False)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 101, in flatten
    self._write(msg)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 129, in _write
    self._dispatch(msg)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 155, in _dispatch
    meth(msg)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 270, in _handle_message_delivery_status
    for part in msg.get_payload():
TypeError: iteration over non-sequence

I can provide more samples if necessary. I have searched the tracker for other reports like this, but haven't seen any ...

----------------------------------------------------------------------

>Comment By: Sebastian Hagedorn (shagedorn)
Date: 2008-03-06 09:31

Message:
Logged In: YES 
user_id=309132
Originator: YES

Hi,

thanks for looking into this. To address your questions:

- we are running 2.1.9
- we have a cronjob in cron.daily that does an "unshunt"
- the file I mentioned originally doesn't exist anymore (because it has
been unshunted?)
- the file I attached had this error message:
Mar 05 07:56:19 2008 (4711) SHUNTING:
1167220351.925714+67431f92e260e4da420214c9a343ace677ca1acd
Mar 05 07:56:19 2008 (4711) Uncaught runner exception: iteration over
non-sequence
Mar 05 07:56:19 2008 (4711) Traceback (most recent call last):
  File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 112, in _oneloop
    self._onefile(msg, msgdata)
  File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 170, in _onefile
    keepqueued = self._dispose(mlist, msg, msgdata)
...

So the error is different from what I see with show_qfiles.

- We have not modified the charset of German to UTF-8

Now my question is whether the "unshunt" cron job is evil? I confess I
never properly understood how it works. It seemed to get messages delivered
that were shunted originally, so at some point I set up that cronjob ...

I suppose I should just remove all files that are too old. But: how do I
see the contents when show_qfiles does not work? Are there other tools?

Thanks for all your help!

----------------------------------------------------------------------

Comment By: Mark Sapiro (msapiro)
Date: 2008-03-06 06:55

Message:
Logged In: YES 
user_id=1123998
Originator: NO

I have looked further at the shunt queue entry
1167220351.925714+67431f92e260e4da420214c9a343ace677ca1acd.pck. Here's what
I see.

This message was a password reminder, originally generated by mailman on
May 1, 2006. It was addressed to a user whose preferred language appears to
be Korean as it is 'Content-Type: text/plain, charset="euc-kr"' with
'Subject: =?euc-kr?q?uni-koeln=2Ede_mailing_list_memberships_reminder?='.

Apparently there was some error at that time which caused the message to
be shunted. I am a bit confused about what happened next because the time
stamp on the queue entry and the received_time in the message's metadata
are 1167220351.925714 which is 11:52:31 Dec 27, 2006 +0000.

However, the message metadata says the message was originally in the 'in'
queue, so I think it must have been unshunted to the 'in' queue where it
couldn't be properly processed, causing it to be shunted again in December,
2006.

The question now is what is causing these messages to be reprocessed at
this point, and why would a message that threw an exception in trying to
flatten the message in SpamDetect be shunted with a 14+ month old time
stamp?

In any case, I think the current error with this particular message is due
to it's being unshunted to the wrong queue.


----------------------------------------------------------------------

Comment By: Mark Sapiro (msapiro)
Date: 2008-03-05 17:58

Message:
Logged In: YES 
user_id=1123998
Originator: NO

The file you attached appears to be a password reminder with a lot of
garbled unicode. Have you changed the characterset for the german language
to utf-8? There is some issue with the templates/de/cronpass.txt template
or a list specific version of it.


----------------------------------------------------------------------

Comment By: Mark Sapiro (msapiro)
Date: 2008-03-05 17:47

Message:
Logged In: YES 
user_id=1123998
Originator: NO

The queue entry you attached is not the one from the reported traceback
and it seems to have a different problem. When I attempt to look at it with
bin/dumpdb or bin/show_qfiles, I get UnicodeEncodeError: 'ascii' codec
can't encode character u'\uc774' in position 0: ordinal not in range(128)
in a different part of Generator.py.

I see that you see the same.

What is your Mailman version?

Also, if I look at the contents of the queue entry, it appears to be a
password reminder which is strange.

I don't have time to look at this further right now, but if you can, can
you find the 1167220351.925714+8af35bc9b49ca21d542d7fac99bdb87a688619ef.pck
and attach that.

Also, is the error in Mailman's error log for the
1167220351.925714+67431f92e260e4da420214c9a343ace677ca1acd.pck file the
same UnicodeEncodeError that you get from show_qfiles and dumpdb, or is it
the TypeError as in the original report?

----------------------------------------------------------------------

Comment By: Sebastian Hagedorn (shagedorn)
Date: 2008-03-05 17:22

Message:
Logged In: YES 
user_id=309132
Originator: YES

OK, I have attached the file. Its name is different, but I guess it's the
same(?):

# grep 1167220351.925714 /var/log/mailman/error
Mar 02 07:37:24 2008 (4711) SHUNTING:
1167220351.925714+55bb5ab698f528a8061d4ef867a634abbed6769e
Mar 03 07:36:03 2008 (4711) SHUNTING:
1167220351.925714+d29bf7b17b742fca8b0b71c09c1e52ebcf08620e
Mar 04 07:50:44 2008 (4711) SHUNTING:
1167220351.925714+8af35bc9b49ca21d542d7fac99bdb87a688619ef
Mar 05 07:56:19 2008 (4711) SHUNTING:
1167220351.925714+67431f92e260e4da420214c9a343ace677ca1acd

Neither show_qfiles nor dbdump works:

/usr/lib/mailman/bin/show_qfiles
/var/spool/mailman/shunt/1167220351.925714+67431f92e260e4da420214c9a343ace677ca1acd.pck

====================>
/var/spool/mailman/shunt/1167220351.925714+67431f92e260e4da420214c9a343ace677ca1acd.pck
Traceback (most recent call last):
  File "/usr/lib/mailman/bin/show_qfiles", line 95, in ?
    main()
  File "/usr/lib/mailman/bin/show_qfiles", line 88, in main
    sys.stdout.write(msg.as_string())
  File "/usr/lib/mailman/pythonlib/email/Message.py", line 135, in
as_string
    g.flatten(self, unixfrom=unixfrom)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 101, in
flatten
    self._write(msg)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 129, in
_write
    self._dispatch(msg)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 155, in
_dispatch
    meth(msg)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 198, in
_handle_text
    self._fp.write(payload)
UnicodeError: ASCII encoding error: ordinal not in range(128)
[root at lvr13 Mailman]# /usr/lib/mailman/bin/dumpdb
/var/spool/mailman/shunt/1167220351.925714+67431f92e260e4da420214c9a343ace677ca1acd.pck

[----- start pickle file -----]
<----- start object 1 ----->
Traceback (most recent call last):
  File "/usr/lib/mailman/bin/dumpdb", line 159, in ?
    msg = main()
  File "/usr/lib/mailman/bin/dumpdb", line 149, in main
    pp.pprint(obj)
  File "/usr/lib/python2.2/pprint.py", line 110, in pprint
    self.__stream.write(self.pformat(object) + "\n")
  File "/usr/lib/python2.2/pprint.py", line 114, in pformat
    self.__format(object, sio, 0, 0, {}, 0)
  File "/usr/lib/python2.2/pprint.py", line 136, in __format
    rep = self.__repr(object, context, level - 1)
  File "/usr/lib/python2.2/pprint.py", line 200, in __repr
    self.__depth, level)
  File "/usr/lib/python2.2/pprint.py", line 287, in _safe_repr
    rep = `object`
  File "/usr/lib/mailman/Mailman/Message.py", line 51, in __repr__
    return self.__str__()
  File "/usr/lib/mailman/pythonlib/email/Message.py", line 121, in
__str__
    return self.as_string(unixfrom=True)
  File "/usr/lib/mailman/pythonlib/email/Message.py", line 135, in
as_string
    g.flatten(self, unixfrom=unixfrom)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 101, in
flatten
    self._write(msg)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 129, in
_write
    self._dispatch(msg)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 155, in
_dispatch
    meth(msg)
  File "/usr/lib/mailman/pythonlib/email/Generator.py", line 198, in
_handle_text
    self._fp.write(payload)
UnicodeError: ASCII encoding error: ordinal not in range(128)



----------------------------------------------------------------------

Comment By: Sebastian Hagedorn (shagedorn)
Date: 2008-03-05 17:19

Message:
Logged In: YES 
user_id=309132
Originator: YES

File Added: 1167220351.925714+67431f92e260e4da420214c9a343ace677ca1acd.pck

----------------------------------------------------------------------

Comment By: Mark Sapiro (msapiro)
Date: 2008-03-05 17:14

Message:
Logged In: YES 
user_id=1123998
Originator: NO

There is a problem with the message. Try running bin/show_qfiles or if
that doesn't work, bin/dumpdb on one or more of the shunted files (e.g.
qfiles/shunt/1167220351.925714+8af35bc9b49ca21d542d7fac99bdb87a688619ef.pck).

In any case, can you send me this file or attach it to this report.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1908112&group_id=103


More information about the Mailman-coders mailing list