[Spambayes] IMAPFilter training issues (WAS imapfilter mangling headers!)

[Oliver Maunder]

Tim Stone - Four Stones Expressions wrote:

>4/22/2003 6:43:06 AM, Oliver Maunder <lists at olivermaunder.co.uk> wrote:
>  
>
>Well, let's check a couple of things.  I've had a bit of trouble with 
>*thinking* I had the latest cvs version, but that actually not being the case.  
>The correct current version of imapfilter.py is 1.29.  Can you check that?
>  
>
I've been caught that way too! But I deleted everything in my spambayes 
folder except bayescustomize.ini before checking out the latest code. 
I'm now using 1.3.

>Also, are you certain that you've set the spam_train_folders and 
>ham_train_folders options to the correct folder name(s)?  Believe me, I'm not 
>trying to insult your intelligence, but just for my own comfort level, can you 
>check to be sure those values are correct in your bayescustomize.ini file (not 
>just on the config page).
>  
>
Pretty certain - I created the .ini file manually. When I ran the config 
page, it picked up all the correct values out of the file. Also the 
config page managed to show all my folders on the IMAP server, so it's 
obviously able to connect.

>In the meantime, I'll check in a version of imap filter that gives a bit more 
>verbose output while training.  If all the above stuff is good, then use this 
>version, capture the output, and attach it to your response, if you will.  
>
It's a pleasure, although not overly enlightening. I added a message 
count to the output:

C:\Development\SpamBayes\spambayes>imapfilter.py  -v -t
Loading database hammie.db... Done.
Training
   Training ham folder INBOX.spambayes.ham-train
Total messages in INBOX.spambayes.ham-train: 122
       0 trained.
   Training spam folder INBOX.spambayes.spam-train
Total messages in INBOX.spambayes.spam-train: 95
       0 trained.
Training took 0.516000032425 seconds, 0 messages were trained

There's a combined total of over 200 messages in those folders, so half 
a second seems a bit quick to do anything with them!

UPDATE:  Just done some more debugging, and the regex match seems to be 
failing in IMAPFolder.keys(). My server returns responses like
83 (FLAGS (\Seen) UID 131)

whereas the regex is looking for
r"[0-9]+ \(UID ([0-9]+) FLAGS \(([\\\w]*)\)\)"

which seems to have things in a different order.

I'm getting a growing feeling that my ISPs mailserver is very badly 
behaved indeed!

Any regular expression experts out there? I might just be able to mangle 
it enough so it matches *my* server's responses, but that would break 
compatibility with everyone else.

Olly