Mar 26 21:56:41 2002 (558) Uncaught runner exception: Continuation line seen before first header Mar 26 21:56:41 2002 (558) Traceback (most recent call last): File "/home/mailman/Mailman/Queue/Runner.py", line 105, in __oneloop self.__onefile(msg, msgdata) File "/home/mailman/Mailman/Queue/Runner.py", line 155, in __onefile keepqueued = self._dispose(mlist, msg, msgdata) File "/home/mailman/Mailman/Queue/IncomingRunner.py", line 111, in _dispose status = self._dopipeline(mlist, msg, msgdata, pipeline) File "/home/mailman/Mailman/Queue/IncomingRunner.py", line 134, in _dopipeline sys.modules[modname].process(mlist, msg, msgdata) File "/home/mailman/Mailman/Handlers/Tagger.py", line 47, in process matchlines.extend(scanbody(msg, mlist.topics_bodylines_limit)) File "/home/mailman/Mailman/Handlers/Tagger.py", line 101, in scanbody msg = email.message_from_string(EMPTYSTRING.join(lines)) File "/home/mailman/pythonlib/email/__init__.py", line 36, in message_from_string return _Parser(_class).parsestr(s) File "/home/mailman/pythonlib/email/Parser.py", line 45, in parsestr return self.parse(StringIO(text)) File "/home/mailman/pythonlib/email/Parser.py", line 40, in parse self._parseheaders(root, fp) File "/home/mailman/pythonlib/email/Parser.py", line 69, in _parseheaders raise Errors.HeaderParseError( HeaderParseError: Continuation line seen before first header
Now, I'd believe there was something wrong with the header (although mm ought to catch this more elegantly), except the user had just sent it to list1@myserver,list2@myserver, and it made it to *list2*, it just won't go through to list1. I unshunted it, and it failed again the same way.
So how in heck did it get to list2?
Barry, I saved the copy I got (I'm on both lists), and the shunt files, if you want them; there's nothing private in the message.
At 11:16 PM 3/26/02 -0500, Ron Jarrell wrote:
Mar 26 21:56:41 2002 (558) Uncaught runner exception: Continuation line seen before first header
It behooves me to point out that this is with 2.1b1 + today's cvs activity, down through barry's double commit of add_user.
Ok, I eliminated differences between the list configurations (one had multiple languages enabled, and topics, the other didn't.) Getting their configs to match, other than the names of the lists and such didn't change the process enough to cure the error. I did find one odd thing.
The messages was delivered to my mm box in the same SMTP transaction, although, of course, it started 2 seperate pipes up for the delivery.
The header on the one I received through list 2 includes
Received: from dagger.cc.vt.edu (IDENT:mirapoint@dagger-lb.cc.vt.edu [10.1.1.11]) by lennier.cc.vt.edu (8.11.4/8.11.4) with ESMTP id g2R2udI145446; Tue, 26 Mar 2002 21:56:39 -0500 (EST)
(Where that first whitespace on the continuation lines really is a space.)
The header on the one in my shunt pickle is:
Received: from dagger.cc.vt.edu (IDENT:mirapoint@dagger-lb.cc.vt.edu [10.1.1.11]) by lennier.cc.vt.edu (8.11.4/8.11.4) with ESMTP id g2R2udI145446; Tue, 26 Mar 2002 21:56:39 -0500 (EST)
Where that initial whitespace is a tab. I'm guessing someone, somewhere, rewrote my received lines after it hit mailman... Otherwise why are they folded differently? But, anyway, unless mm is bitching about a continuation line starting with a [, I'm at a loss why two messages that were identical until injected into mm delivery would react so differently.
"RJ" == Ron Jarrell <jarrell@vt.edu> writes:
RJ> Now, I'd believe there was something wrong with the header
RJ> (although mm ought to catch this more elegantly), except the
RJ> user had just sent it to list1@myserver,list2@myserver, and it
RJ> made it to *list2*, it just won't go through to list1. I
RJ> unshunted it, and it failed again the same way.
RJ> So how in heck did it get to list2?
RJ> Barry, I saved the copy I got (I'm on both lists), and the
RJ> shunt files, if you want them; there's nothing private in the
RJ> message.Yes, please do send them to me! -Barry
At 02:45 PM 3/27/02 -0500, Barry A. Warsaw wrote:
"RJ" == Ron Jarrell <jarrell@vt.edu> writes:
RJ> Now, I'd believe there was something wrong with the header RJ> (although mm ought to catch this more elegantly), except the RJ> user had just sent it to list1@myserver,list2@myserver, and it RJ> made it to *list2*, it just won't go through to list1. I RJ> unshunted it, and it failed again the same way.
RJ> So how in heck did it get to list2?
RJ> Barry, I saved the copy I got (I'm on both lists), and the RJ> shunt files, if you want them; there's nothing private in the RJ> message.
Yes, please do send them to me! -Barry
Hey, Barry? It's the topic processor causing it. I got a second message to that same list that generated the same error. I'd tried turning off topics before, since that was one of the two differences between the working, and non working, list. But when I did that, I didn't *delete* the topic, I just disabled it. (Although, if topics are disabled, that really should skip all the topic header scanning... Apparently it doesn't.)
I tried disabling topics, *and* deleting my one trial topic. Suddenly the two messages (the one I sent you, and the latest one) would unshunt.
The only trial topic I had was, for fooling around purposes, on that list was (from config_list):
topics_enabled = 1 topics_bodylines_limit = 5 topics = [('Illuminati', '[fF]nord', 'Illuminated Messages', 0)]
At 01:04 AM 3/29/02 -0500, Ron Jarrell wrote:
At 02:45 PM 3/27/02 -0500, Barry A. Warsaw wrote:
> "RJ" == Ron Jarrell <jarrell@vt.edu> writes:
RJ> Now, I'd believe there was something wrong with the header RJ> (although mm ought to catch this more elegantly), except the RJ> user had just sent it to list1@myserver,list2@myserver, and it RJ> made it to *list2*, it just won't go through to list1. I RJ> unshunted it, and it failed again the same way.
RJ> So how in heck did it get to list2?
RJ> Barry, I saved the copy I got (I'm on both lists), and the RJ> shunt files, if you want them; there's nothing private in the RJ> message.
Yes, please do send them to me! -Barry
Hey, Barry? It's the topic processor causing it. I got a second message to that same list that generated the same error. I'd tried turning off topics before, since that was one of the two differences between the working, and non working, list. But when I did that, I didn't *delete* the topic, I just disabled it. (Although, if topics are disabled, that really should skip all the topic header scanning... Apparently it doesn't.)
Barry, did any of your recent fixes address this one? I've been avoiding putting back topics for now.
"RJ" == Ron Jarrell <jarrell@vt.edu> writes:
RJ> Hey, Barry? It's the topic processor causing it. I got a
RJ> second message to that same list that generated the same
RJ> error. I'd tried turning off topics before, since that was
RJ> one of the two differences between the working, and non
RJ> working, list. But when I did that, I didn't *delete* the
RJ> topic, I just disabled it. (Although, if topics are disabled,
RJ> that really should skip all the topic header
RJ> scanning... Apparently it doesn't.)
RJ> I tried disabling topics, *and* deleting my one trial topic.
RJ> Suddenly the two messages (the one I sent you, and the latest
RJ> one) would unshunt.
RJ> The only trial topic I had was, for fooling around purposes,
RJ> on that list was (from config_list):
RJ> topics_enabled = 1 topics_bodylines_limit = 5 topics =
RJ> [('Illuminati', '[fF]nord', 'Illuminated Messages', 0)]I've been trying to reproduce this with current cvs and am unable to. I'm not sure that the topic processor's logic is totally correct, but I also can't provoke the bugs you describe.
Can you please do two things: try to update to current cvs, do a "./config.status ; make install ; cd $prefix ; bin/mailmanctl restart"
If you still get the bugs, please step me through the recipe you're using to cause the bug.
In the meantime, I'll try to convince myself about the logic of the topic processor.
-Barry
At 01:57 AM 4/11/02 -0400, Barry A. Warsaw wrote:
"RJ" == Ron Jarrell <jarrell@vt.edu> writes:
RJ> Hey, Barry? It's the topic processor causing it. I got a RJ> second message to that same list that generated the same RJ> error. I'd tried turning off topics before, since that was RJ> one of the two differences between the working, and non RJ> working, list. But when I did that, I didn't *delete* the RJ> topic, I just disabled it. (Although, if topics are disabled, RJ> that really should skip all the topic header RJ> scanning... Apparently it doesn't.) RJ> I tried disabling topics, *and* deleting my one trial topic. RJ> Suddenly the two messages (the one I sent you, and the latest RJ> one) would unshunt. RJ> The only trial topic I had was, for fooling around purposes, RJ> on that list was (from config_list): RJ> topics_enabled = 1 topics_bodylines_limit = 5 topics = RJ> [('Illuminati', '[fF]nord', 'Illuminated Messages', 0)]I've been trying to reproduce this with current cvs and am unable to. I'm not sure that the topic processor's logic is totally correct, but I also can't provoke the bugs you describe.
Can you please do two things: try to update to current cvs, do a "./config.status ; make install ; cd $prefix ; bin/mailmanctl restart"
If you still get the bugs, please step me through the recipe you're using to cause the bug.
In the meantime, I'll try to convince myself about the logic of the topic processor.
I rebuild like that regularly... As I reported, it happened again a couple of days ago. That shut pickle set I sent you contains a message that triggers it every time. If I turn topics back on and install a topic (I used the one above, again), *eventually* a message will trigger the problem. Not every message does, just some.
If I unshunt it, i'll just get repeated continuation before header errors. If I delete the topic, and turn off topics (just turning off topics wasn't enough the last time, I had to also delete the topic list) it'll unshunt just fine.
Hence why I'm blaming the topic processor!
At 01:57 AM 4/11/02 -0400, Barry A. Warsaw wrote:
If you still get the bugs, please step me through the recipe you're using to cause the bug.
Ok. I updated cvs. I configed again, did a make, a make install. All the mailman daemons were stopped before, and started after. All the .pyc files properly generated. I'm about as current as it's possible to get. I turned topics back on. I put back my test topic. I used my archived copy of the shunt files that I sent you some time ago, and unshunted them back into the queue. They failed again with the same error...
Apr 11 09:04:44 2002 (11160) Uncaught runner exception: Continuation line seen before first header Apr 11 09:04:44 2002 (11160) Traceback (most recent call last): File "/home/mailman/Mailman/Queue/Runner.py", line 105, in __oneloop self.__onefile(msg, msgdata) File "/home/mailman/Mailman/Queue/Runner.py", line 155, in __onefile keepqueued = self._dispose(mlist, msg, msgdata) File "/home/mailman/Mailman/Queue/IncomingRunner.py", line 129, in _dispose status = self._dopipeline(mlist, msg, msgdata, pipeline) File "/home/mailman/Mailman/Queue/IncomingRunner.py", line 152, in _dopipeline sys.modules[modname].process(mlist, msg, msgdata) File "/home/mailman/Mailman/Handlers/Tagger.py", line 47, in process matchlines.extend(scanbody(msg, mlist.topics_bodylines_limit)) File "/home/mailman/Mailman/Handlers/Tagger.py", line 101, in scanbody msg = email.message_from_string(EMPTYSTRING.join(lines)) File "/home/mailman/pythonlib/email/__init__.py", line 36, in message_from_string return _Parser(_class).parsestr(s) File "/home/mailman/pythonlib/email/Parser.py", line 45, in parsestr return self.parse(StringIO(text)) File "/home/mailman/pythonlib/email/Parser.py", line 40, in parse self._parseheaders(root, fp) File "/home/mailman/pythonlib/email/Parser.py", line 76, in _parseheaders raise Errors.HeaderParseError( HeaderParseError: Continuation line seen before first header
As a reminder - this is a message that went to list1,list2, made it to list1, but kept bouncing to list2. The only real difference in the two list configs is one had topics enabled, and one didn't. If I delete the topic and turn off topic processing, this note will unshunt and post to list2 as well. Here's the relevant section from config_list for list2:
topics_enabled = 1 topics_bodylines_limit = 5 topics = [('Iluminatti', '[F][f]nord', 'Illuminated Messages', 0)]
I'm guessing that there's something wrong in the way Tagger calls Parser in this case... I'll eyeball the code again myself; maybe a fresh pair of eyes will catch something you're missing cause you've been staring at it too long.
At 09:13 AM 4/11/02 -0400, Ron Jarrell wrote:
I'm guessing that there's something wrong in the way Tagger calls Parser in this case... I'll eyeball the code again myself; maybe a fresh pair of eyes will catch something you're missing cause you've been staring at it too long.
Hah!
This is my morning for epiphanies. I suppose that it helps that this turns out to be very similar to a problem the mirapoint scanners were having this morning, and I already had the brainstorm that figured out their continuation header processing was broken.
You've got a logic flaw. Let me guess, you don't indent much when you write prose, do you :-)?
You allow through lines that start with a whitespace into the new message object that you're building to parse the headers out of, since they're likely a continuation header. But they're likely not a continuation header *in the body of the message*!
Send a message to a list with topics defined. (See my previous note about the bug in topics_enabled - only the gui uses it, Tagger keys on topics defined.
Indent the first line. Like: Hi there
You should get the error when Parser hits it before it hits anything else in the new "header".
Tagger.py:96 says:
if line[0] not in ' \t' and line.find(':') < 0:
So any line that starts with a whitespace gets bundled into the test, even if the *previous* line wasn't a possible header line.
If you want to support continuation lines on pseudo-headers in the body, you either need to maintain state like Parser does, to decide whether it's really a continuation, or just an indented line, or just bag that test all together, and only accept lines with a ":", and not support continuation lines in the pseudo-headers.
At 09:38 AM 4/11/02 -0400, Ron Jarrell wrote:
if line[0] not in ' \t' and line.find(':') < 0:
Ok. My patch (because I'm lazy enough to decide that supporting pseudo continuations is not nearly important enough to worry about) is:
Index: Tagger.py
RCS file: /cvsroot/mailman/mailman/Mailman/Handlers/Tagger.py,v retrieving revision 2.3 diff -r2.3 Tagger.py 33c33 < if not mlist.topics:
if not mlist.topics_enabled:
96c96 < if line[0] not in ' \t' and line.find(':') < 0:
if line[0] in ' \t' or line.find(':') < 0:
This does work, and the messages now unshunt just hunky-dory.
"RJ" == Ron Jarrell <jarrell@vt.edu> writes:
RJ> This is my morning for epiphanies.Yay! I'm glad somebody's getting them. ;)
RJ> You've got a logic flaw. Let me guess, you don't indent much
RJ> when you write prose, do you :-)?Heh, no.
I think you've nailed the problem right on the head, thanks! Of course, I just have to have a more elaborate fix, which sweater-threaded a few other bugs, but I think I now have this working. CVS commits to take place momentarily.
Thanks Ron! -Barry
"RJ" == Ron Jarrell <jarrell@vt.edu> writes:
>> working. CVS commits to take place momentarily.
RJ> Did you mean to leave the syslog('debug',foo) lines in Tagger
RJ> files in cvs?Of course! I always do everythign on purpose. Oh alright, I'll remove them.
SIGH.
:) -Barry
Ok, well, one thing I see...
Tagger.py:33
Shouldn't if not mlist.topics: be if not mlist.topics_enabled:
? Because this is why I'm seeing the "must delete topics" to fix the problem. If you disable topics, but have topic defined, you still end up scanning for them. If they're not enabled, we shouldn't bother doing the work, even if there are some defined.
participants (2)
-
barry＠zope.com -
Ron Jarrell