[Spambayes] Bug in SpamBayes - It won't classify all new mail from a pop server that has old mail saved on it

Lanny Trager ltrag at optonline.net
Thu Sep 11 22:18:42 EDT 2003


Sorry, this was previosly sent as a reply to anlther thread by accident:


This is a followup on a problem that I have been having since I began using 
Kmail a few weeks ago.  Seen before with the Subject: Classification not 
always added to the header 

Briefly, I am running:

spambayes-1.0a5 (getting mail from two different pop servers)
RedHat 9
Kmail

After properly shutting down SpamBayes, Kmail and then my machine, the next 
morning SpamBayes does not classify the first few e-mails from one of the 3 
pop servers that I use.

After, much investigation, I discovered that Kmail places several files in 
"/home/myusername/.kde/share/apps/kmail" (myusername is a munge of the actual  
user name).   These files are called (after SpamBayes installation) 
mungeduser at localhost:port#_that_email_client_points_to, there is one of these 
files for each pop server or in the case of one of the pop servers that I 
have two accounts on there are two of these files each pointing to the same 
server and port but with a different user name, reflecting my user names on 
the pop server.

The purpose of these files is to keep track of what you want to download later 
and what is to be kept on the server.  This is reflected by two commands: 
downloadLater=
seenUidList=

In my case there are perhaps 80 e-mails that I have chosen to keep on one of 
the servers, so after seenUidList= there are a series of numbers separated by 
commas.  I don't think that it is a coincidence that this is the server from 
which e-mails are not being classified all of the time.  (Usually the problem 
occurs only the first one or two times that I get my e-mail after starting 
SpamBayes and kmail in the morning.

As an experiment I moved these files to another directory and recreated them, 
leaving the command in Kmail to leave the e-mail on the server.  By 
recreating the files I had emptied them so that Kmail was not aware of what 
was on the server.  

I did this twice, the first time Kmail/SpamBayes proceeded to download the 
mail from the server but hung after the first few e-mails were downloaded.  
The messages that appeared in the terminal were:

Traceback (most recent call last):
  File "pop3proxy.py", line 433, in onRetr
    msg.setPayload(messageText)
  File "spambayes/message.py", line 177, in setPayload
    prs._parseheaders(self, fp)
  File "/usr/lib/python2.2/email/Parser.py", line 128, in _parseheaders
    raise Errors.HeaderParseError(
HeaderParseError: Not a header, not a continuation: ``the first time that I've 
been on my windows machine in many months so I ''
 
Traceback (most recent call last):
  File "pop3proxy.py", line 433, in onRetr
    msg.setPayload(messageText)
  File "spambayes/message.py", line 177, in setPayload
    prs._parseheaders(self, fp)
  File "/usr/lib/python2.2/email/Parser.py", line 128, in _parseheaders
    raise Errors.HeaderParseError(
HeaderParseError: Not a header, not a continuation: 
``ZXJlIENhdGFsb2cgTWFuYWdlci4gIFJlc3VsdGVkIGluIGN1c3RvbWVycyBnZXR0aW5nIG5ldyBj''
 
Traceback (most recent call last):
  File "pop3proxy.py", line 433, in onRetr
    msg.setPayload(messageText)
  File "spambayes/message.py", line 177, in setPayload
    prs._parseheaders(self, fp)
  File "/usr/lib/python2.2/email/Parser.py", line 128, in _parseheaders
    raise Errors.HeaderParseError(
HeaderParseError: Not a header, not a continuation: 
``AAAAABYHAAAAAAAAPAoAAAAAAAAAAAAAAAAAANAHAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA''
 
Traceback (most recent call last):
  File "pop3proxy.py", line 433, in onRetr
    msg.setPayload(messageText)
  File "spambayes/message.py", line 177, in setPayload
    prs._parseheaders(self, fp)
  File "/usr/lib/python2.2/email/Parser.py", line 128, in _parseheaders
    raise Errors.HeaderParseError(
HeaderParseError: Not a header, not a continuation: 
``AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABH''
 
Traceback (most recent call last):
  File "pop3proxy.py", line 433, in onRetr
    msg.setPayload(messageText)
  File "spambayes/message.py", line 177, in setPayload
    prs._parseheaders(self, fp)
  File "/usr/lib/python2.2/email/Parser.py", line 110, in _parseheaders
    raise Errors.HeaderParseError(
HeaderParseError: Continuation line seen before first header
 
error: uncaptured python exception, closing channel <__main__.ServerLineReader 
connected at 0x83580a4> (exceptions.ValueError:unpack list of wrong size 
[/usr/lib/python2.2/asyncore.py|poll|99] 
[/usr/lib/python2.2/asyncore.py|handle_read_event|396] 
[/usr/lib/python2.2/asynchat.py|handle_read|130] 
[pop3proxy.py|found_terminator|146] [pop3proxy.py|onServerLine|214] 
[pop3proxy.py|onResponse|276] [pop3proxy.py|onTransaction|377] 
[pop3proxy.py|onRetr|509])
Traceback (most recent call last):
  File "pop3proxy.py", line 433, in onRetr
    msg.setPayload(messageText)
  File "spambayes/message.py", line 177, in setPayload
    prs._parseheaders(self, fp)
  File "/usr/lib/python2.2/email/Parser.py", line 128, in _parseheaders
    raise Errors.HeaderParseError(
HeaderParseError: Not a header, not a continuation: 
``Y8bX2Dke4pk7Zj8uT5gOhPUe1UEuGhVomj3e2MDH1qbWHe5fnferSRvuBO5Cvesq7s7nDMkWVbnA''
 
Traceback (most recent call last):
  File "pop3proxy.py", line 433, in onRetr
    msg.setPayload(messageText)
  File "spambayes/message.py", line 177, in setPayload
    prs._parseheaders(self, fp)
  File "/usr/lib/python2.2/email/Parser.py", line 128, in _parseheaders
    raise Errors.HeaderParseError(
HeaderParseError: Not a header, not a continuation: 
``Lh1rfy3X88xAnKiqUy/dKACBFbtRHB1V7UZc4Rm4qoOhLoDBn6pfOWZnUzY5PKviPNvr/9BRe6Pb''
 
Traceback (most recent call last):
  File "pop3proxy.py", line 433, in onRetr
    msg.setPayload(messageText)
  File "spambayes/message.py", line 177, in setPayload
    prs._parseheaders(self, fp)
  File "/usr/lib/python2.2/email/Parser.py", line 128, in _parseheaders
    raise Errors.HeaderParseError(
HeaderParseError: Not a header, not a continuation: 
``alt="Careers"></a><!--end_leftnav --> ''
 
Traceback (most recent call last):
  File "pop3proxy.py", line 433, in onRetr
    msg.setPayload(messageText)
  File "spambayes/message.py", line 177, 

I stopped both SpamBayes and Kmail.

I Restarted SpamBayes and Kmail and tried the experiment again.  It hung again 
but this time I decided to let it go, after a few minutes it continued to 
download the emails but did not add any classifications to the rest of the 
e-mails.  (It did add classifications to the first few e-mails.)

p3proxy.py", line 433, in onRetr
    msg.setPayload(messageText)
  File "spambayes/message.py", line 177, in setPayload
    prs._parseheaders(self, fp)
  File "/usr/lib/python2.2/email/Parser.py", line 110, in _parseheaders
    raise Errors.HeaderParseError(
HeaderParseError: Continuation line seen before first header
 
Traceback (most recent call last):
  File "pop3proxy.py", line 433, in onRetr
    msg.setPayload(messageText)
  File "spambayes/message.py", line 177, in setPayload
    prs._parseheaders(self, fp)
  File "/usr/lib/python2.2/email/Parser.py", line 128, in _parseheaders
    raise Errors.HeaderParseError(
HeaderParseError: Not a header, not a continuation: ``we can't afford to be. 
"Is your ISP blocking us? Is their ISP blocking us?''
 
Traceback (most recent call last):
  File "pop3proxy.py", line 433, in onRetr
    msg.setPayload(messageText)
  File "spambayes/message.py", line 177, in setPayload
    prs._parseheaders(self, fp)
  File "/usr/lib/python2.2/email/Parser.py", line 128, in _parseheaders
    raise Errors.HeaderParseError(
HeaderParseError: Not a header, not a continuation: ``<ul>''
 
Traceback (most recent call last):
  File "pop3proxy.py", line 433, in onRetr
    msg.setPayload(messageText)
  File "spambayes/message.py", line 177, in setPayload
    prs._parseheaders(self, fp)
  File "/usr/lib/python2.2/email/Parser.py", line 128, in _parseheaders
    raise Errors.HeaderParseError(
HeaderParseError: Not a header, not a continuation: 
``sjtVETUus2U8AsHqb18zVUHyhCPhHxY6Is53zU6VeEcHGSsdTQOf+lo4VnpvlpAIwqBxUfFRqpXx''
 
Traceback (most recent call last):
  File "pop3proxy.py", line 433, in onRetr
    msg.setPayload(messageText)
  File "spambayes/message.py", line 177, in setPayload
    prs._parseheaders(self, fp)
  File "/usr/lib/python2.2/email/Parser.py", line 128, in _parseheaders
    raise Errors.HeaderParseError(
HeaderParseError: Not a header, not a continuation: 
``eGcDwMGQdb8BztWiF1zgSOf0wGup9IO/POFQv+mwteufEB7WyBPvMcazTvGuP27jYOf4szX+''
 
Traceback (most recent call last):
  File "pop3proxy.py", line 433, in onRetr
    msg.setPayload(messageText)
  File "spambayes/message.py", line 177, in setPayload
    prs._parseheaders(self, fp)
  File "/usr/lib/python2.2/email/Parser.py", line 128, in _parseheaders
    raise Errors.HeaderParseError(
HeaderParseError: Not a header, not a continuation: ``) Breathing Problems?''
 
Traceback (most recent call last):
  File "pop3proxy.py", line 433, in onRetr
    msg.setPayload(messageText)
  File "spambayes/message.py", line 177, in setPayload
    prs._parseheaders(self, fp)
  File "/usr/lib/python2.2/email/Parser.py", line 128, in _parseheaders
    raise Errors.HeaderParseError(
HeaderParseError: Not a header, not a continuation: ``else''
 
Traceback (most recent call last):
  File "pop3proxy.py", line 433, in onRetr
    msg.setPayload(messageText)
  File "spambayes/message.py", line 177, in setPayload
    prs._parseheaders(self, fp)
  File "/usr/lib/python2.2/email/Parser.py", line 128, in _parseheaders
    raise Errors.HeaderParseError(
HeaderParseError: Not a header, not a continuation: 
``............................................................''
 
Traceback (most recent call last):
  File "pop3proxy.py", line 433, in onRetr
    msg.setPayload(messageText)
  File "spambayes/message.py", line 177, in setPayload
    prs._parseheaders(self, fp)
  File "/usr/lib/python2.2/email/Parser.py", line 128, in _parseheaders
    raise Errors.HeaderParseError(
HeaderParseError: Not a header, not a continuation: ``//-->''

After this last experiment, I left Kmail and SpamBayes running and all e-mails 
that arrived subsequent to the experiment were classified.

While I might be willing to remove the mail from the server in order to derive 
the benefits of SpamBayes, I would really rather not.

I think that SpamBayes ought to be capable of dealing with mail left on a pop 
server.  For folks like me who retrieve their mail using more than one 
machine this is essential.  When I am on the road I want to read my mail, but 
when I get back home I want to be able to download that same mail to my home 
machine.

Lanny




More information about the Spambayes mailing list