[spambayes-bugs] [ spambayes-Support Requests-941596 ] sb_imapfilter.py not adding headers / moving messages

SourceForge.net noreply at sourceforge.net
Mon Apr 26 22:40:20 EDT 2004


Support Requests item #941596, was opened at 2004-04-24 20:55
Message generated for change (Comment added) made by warehauz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498104&aid=941596&group_id=61702

Category: None
Group: None
Status: Open
Priority: 5
Submitted By: Joe (warehauz)
Assigned to: Nobody/Anonymous (nobody)
Summary: sb_imapfilter.py not adding headers / moving messages

Initial Comment:
I've got 1.0b1 installed on a Linux server and got
everything configured for IMAP usage, and have trained
a few dozen spams & hams.  Everything appears to be
going well, and when I run "sb_imapfilter.py -t -c" the
output looks fine, but the classification isn't fully
working - messages aren't moved, and they don't get all
the headers added they're supposed to have.

Basically, when training (i.e. I manually put a message
in the spam_train_folders), it will correctly process
the message, and move it to the
move_trained_spam_to_folder folder.  After manually
training I looked up a few words in the database via
the web interface to make sure everything is working
properly, and it does indeed appear to be.

But, when classifying, the output will say for example:

Classified 0 ham, 1 spam, and 0 unsure.

The message was actually spam, so that's fine - but it
left it in the inbox, and didn't add a classification
or score header (I've turned those options on).  So
while the classification is picking the right option,
it's not doing anything with the messages.  The message
remains in the original folder, regardless of it's
classification, and only gets the X-Spambayes-Mailid
header.

I also noticed a minor problem with the training as
well, but since the messages are being moved to the
proper folder, and the tokens are being added to the
training database properly, I'm not concerned about it.
 I've set include_trained to True, so it should be
adding a header to indicate it's trained on that
message - but it doesn't.  In fact, when it moves the
message to the move_trained_*_to_folder folder, even
the X-Spambayes-Mailid is gone.  Since it's doing the
training fine though, I'm not worried - I just suspect
it's related to my bigger classification problems.

Is this a known problem?  Anything I can do to
fix/workaround this?  It almost seems to me like it's
deleting the wrong message (as I understand it, it's
supposed to create a second copy of the message with
the new headers, and delete the original).  FYI, the
IMAP server appears to be Courier-IMAP.

Here's some lengthy output from a "sb_imapfilter.py -t
-c -v -i4" that found & supposedly classified a spam
message, but ended up leaving it in the inbox without
the classification headers etc:

======

SpamBayes IMAP Filter Version 0.3 (April 2004)
and engine SpamBayes Engine Version 0.3 (January 2004).

Loading database /usr/local/spambayes/hammie.db... Done.
Training
   Training ham folder INBOX.HAM
       0 trained.
   Training spam folder INBOX.SPAM
       0 trained.
Training took 0.0049 seconds, 0 messages were trained
Classifying
.*.
Classified 0 ham, 1 spam, and 0 unsure.
Classifying took 0.0689 seconds.

Loading state from /usr/local/spambayes/hammie.db database
/usr/local/spambayes/hammie.db is an existing database,
with 29 spam and 21 ham
  09:01.22 > DMNB1 LOGIN "USERNAME" "***PASSWORD***"
  09:01.22 < DMNB1 OK LOGIN Ok.
  09:01.23 > DMNB2 SELECT INBOX.HAM
  09:01.23 < * FLAGS (\Draft \Answered \Flagged
\Deleted \Seen \Recent)
  09:01.23 < * OK [PERMANENTFLAGS (\* \Draft \Answered
\Flagged \Deleted \Seen)] Limited
  09:01.23 < * 0 EXISTS
  09:01.23 < * 0 RECENT
  09:01.23 < * OK [UIDVALIDITY 1082777941] Ok
  09:01.23 < * OK [MYRIGHTS "acdilrsw"] ACL
  09:01.23 < DMNB2 OK [READ-WRITE] Ok
  09:01.23 > DMNB3 UID SEARCH UNDELETED
  09:01.23 < * SEARCH
  09:01.23 < DMNB3 OK SEARCH done.
  09:01.23 > DMNB4 CLOSE
  09:01.23 < DMNB4 OK mailbox closed.
  09:01.23 > DMNB5 SELECT INBOX.SPAM
  09:01.23 < * FLAGS (\Draft \Answered \Flagged
\Deleted \Seen \Recent)
  09:01.23 < * OK [PERMANENTFLAGS (\* \Draft \Answered
\Flagged \Deleted \Seen)] Limited
  09:01.23 < * 0 EXISTS
  09:01.23 < * 0 RECENT
  09:01.23 < * OK [UIDVALIDITY 1082777941] Ok
  09:01.23 < * OK [MYRIGHTS "acdilrsw"] ACL
  09:01.23 < DMNB5 OK [READ-WRITE] Ok
  09:01.23 > DMNB6 UID SEARCH UNDELETED
  09:01.23 < * SEARCH
  09:01.23 < DMNB6 OK SEARCH done.
  09:01.23 > DMNB7 CLOSE
  09:01.23 < DMNB7 OK mailbox closed.
  09:01.23 > DMNB8 SELECT INBOX.UNSURE
  09:01.23 < * FLAGS (\Draft \Answered \Flagged
\Deleted \Seen \Recent)
  09:01.23 < * OK [PERMANENTFLAGS (\* \Draft \Answered
\Flagged \Deleted \Seen)] Limited
  09:01.23 < * 0 EXISTS
  09:01.23 < * 0 RECENT
  09:01.23 < * OK [UIDVALIDITY 1082778026] Ok
  09:01.23 < * OK [MYRIGHTS "acdilrsw"] ACL
  09:01.23 < DMNB8 OK [READ-WRITE] Ok
  09:01.23 > DMNB9 CLOSE
  09:01.23 < DMNB9 OK mailbox closed.
  09:01.23 > DMNB10 SELECT INBOX
  09:01.23 < * FLAGS (\Draft \Answered \Flagged
\Deleted \Seen \Recent)
  09:01.23 < * OK [PERMANENTFLAGS (\* \Draft \Answered
\Flagged \Deleted \Seen)] Limited
  09:01.23 < * 2 EXISTS
  09:01.23 < * 1 RECENT
  09:01.23 < * OK [UIDVALIDITY 1082778026] Ok
  09:01.23 < * OK [MYRIGHTS "acdilrsw"] ACL
  09:01.23 < DMNB10 OK [READ-WRITE] Ok
  09:01.23 > DMNB11 UID SEARCH UNDELETED
  09:01.23 < * SEARCH 71 72
  09:01.23 < DMNB11 OK SEARCH done.
  09:01.23 > DMNB12 UID FETCH 71 RFC822.HEADER
  09:01.24 < * 1 FETCH (UID 71 RFC822.HEADER {2564}
  09:01.24 read literal size 2564
  09:01.24 < )
  09:01.24 < DMNB12 OK FETCH completed.
  09:01.24 > DMNB13 UID FETCH 72 RFC822.HEADER
  09:01.24 < * 2 FETCH (UID 72 RFC822.HEADER {1968}
  09:01.24 read literal size 1968
  09:01.24 < )
  09:01.24 < DMNB13 OK FETCH completed.
  09:01.24 > DMNB14 UID FETCH 72 (BODY.PEEK[])
  09:01.24 < * 2 FETCH (UID 72 BODY[] {3908}
  09:01.24 read literal size 3908
  09:01.24 < )
  09:01.24 < DMNB14 OK FETCH completed.
  09:01.24 > DMNB15 UID FETCH 72 (FLAGS INTERNALDATE)
  09:01.25 < * 2 FETCH (UID 72 FLAGS (\Recent)
INTERNALDATE "24-Apr-2004 21:07:27 -0400")
  09:01.25 < DMNB15 OK FETCH completed.
  09:01.25 > DMNB16 APPEND INBOX () "24-Apr-2004
21:07:27 -0400" {3945}
  09:01.25 < + OK
  09:01.25 write literal size 3945
  09:01.26 < DMNB16 OK [APPENDUID 1082778026 73] APPEND Ok.
  09:01.26 > DMNB17 UID STORE 72 +FLAGS.SILENT
(\Deleted \Seen)
  09:01.26 < DMNB17 OK STORE completed.
  09:01.26 > DMNB18 UID SEARCH (UNDELETED HEADER
X-Spambayes-MailId 1082855341)
  09:01.26 < * SEARCH
  09:01.26 < * 2 FETCH (FLAGS (\Seen \Deleted \Recent))
  09:01.26 < DMNB18 OK SEARCH done.
  09:01.26 > DMNB19 UID SEARCH RECENT
  09:01.26 < * SEARCH 72
  09:01.26 < DMNB19 OK SEARCH done.
  09:01.28 > DMNB20 UID FETCH 72 (FLAGS INTERNALDATE)
  09:01.28 < * 2 FETCH (UID 72 FLAGS (\Seen \Deleted
\Recent) INTERNALDATE "24-Apr-2004 21:07:27 -0400")
  09:01.28 < DMNB20 OK FETCH completed.
  09:01.29 > DMNB21 APPEND INBOX.SPAM (\Seen \Deleted)
"24-Apr-2004 22:07:20 -0400" {5014}
  09:01.29 < + OK
  09:01.29 write literal size 5014
  09:01.29 < DMNB21 OK [APPENDUID 1082777941 52] APPEND Ok.
  09:01.29 > DMNB22 UID STORE 72 +FLAGS.SILENT
(\Deleted \Seen)
  09:01.30 < DMNB22 OK STORE completed.
  09:01.30 > DMNB23 CLOSE
  09:01.30 < DMNB23 OK mailbox closed.
  09:01.30 > DMNB24 SELECT INBOX.SPAM
  09:01.30 < * FLAGS (\Draft \Answered \Flagged
\Deleted \Seen \Recent)
  09:01.30 < * OK [PERMANENTFLAGS (\* \Draft \Answered
\Flagged \Deleted \Seen)] Limited
  09:01.30 < * 1 EXISTS
  09:01.30 < * 0 RECENT
  09:01.30 < * OK [UIDVALIDITY 1082777941] Ok
  09:01.30 < * OK [MYRIGHTS "acdilrsw"] ACL
  09:01.30 < DMNB24 OK [READ-WRITE] Ok
  09:01.30 > DMNB25 UID SEARCH (UNDELETED HEADER
X-Spambayes-MailId 1082855341)
  09:01.30 < * SEARCH
  09:01.30 < DMNB25 OK SEARCH done.
  09:01.30 > DMNB26 UID SEARCH RECENT
  09:01.30 < * SEARCH
  09:01.30 < DMNB26 OK SEARCH done.
  09:01.30 > DMNB27 UID SEARCH ALL
  09:01.30 < * SEARCH 52
  09:01.30 < DMNB27 OK SEARCH done.
  09:01.30 > DMNB28 SELECT INBOX.SPAM
  09:01.30 < * FLAGS (\Draft \Answered \Flagged
\Deleted \Seen \Recent)
  09:01.30 < * OK [PERMANENTFLAGS (\* \Draft \Answered
\Flagged \Deleted \Seen)] Limited
  09:01.30 < * 1 EXISTS
  09:01.30 < * 0 RECENT
  09:01.30 < * OK [UIDVALIDITY 1082777941] Ok
  09:01.30 < * OK [MYRIGHTS "acdilrsw"] ACL
  09:01.30 < DMNB28 OK [READ-WRITE] Ok
  09:01.30 > DMNB29 EXPUNGE
  09:01.30 < * 1 EXPUNGE
  09:01.30 < * 0 EXISTS
  09:01.30 < * 0 RECENT
  09:01.30 < DMNB29 OK EXPUNGE completed
  09:01.30 > DMNB30 SELECT INBOX.UNSURE
  09:01.30 < * FLAGS (\Draft \Answered \Flagged
\Deleted \Seen \Recent)
  09:01.30 < * OK [PERMANENTFLAGS (\* \Draft \Answered
\Flagged \Deleted \Seen)] Limited
  09:01.30 < * 0 EXISTS
  09:01.30 < * 0 RECENT
  09:01.30 < * OK [UIDVALIDITY 1082778026] Ok
  09:01.30 < * OK [MYRIGHTS "acdilrsw"] ACL
  09:01.30 < DMNB30 OK [READ-WRITE] Ok
  09:01.30 > DMNB31 EXPUNGE
  09:01.30 < DMNB31 OK EXPUNGE completed
  09:01.30 > DMNB32 SELECT INBOX.HAM
  09:01.30 < * FLAGS (\Draft \Answered \Flagged
\Deleted \Seen \Recent)
  09:01.30 < * OK [PERMANENTFLAGS (\* \Draft \Answered
\Flagged \Deleted \Seen)] Limited
  09:01.30 < * 0 EXISTS
  09:01.30 < * 0 RECENT
  09:01.30 < * OK [UIDVALIDITY 1082777941] Ok
  09:01.30 < * OK [MYRIGHTS "acdilrsw"] ACL
  09:01.30 < DMNB32 OK [READ-WRITE] Ok
  09:01.30 > DMNB33 EXPUNGE
  09:01.31 < DMNB33 OK EXPUNGE completed
  09:01.31 > DMNB34 SELECT INBOX.SPAM
  09:01.31 < * FLAGS (\Draft \Answered \Flagged
\Deleted \Seen \Recent)
  09:01.31 < * OK [PERMANENTFLAGS (\* \Draft \Answered
\Flagged \Deleted \Seen)] Limited
  09:01.31 < * 0 EXISTS
  09:01.31 < * 0 RECENT
  09:01.31 < * OK [UIDVALIDITY 1082777941] Ok
  09:01.31 < * OK [MYRIGHTS "acdilrsw"] ACL
  09:01.31 < DMNB34 OK [READ-WRITE] Ok
  09:01.31 > DMNB35 EXPUNGE
  09:01.31 < DMNB35 OK EXPUNGE completed
  09:01.31 > DMNB36 LOGOUT
  09:01.31 < * BYE Courier-IMAP server shutting down
  09:01.31 BYE response: Courier-IMAP server shutting down
  09:01.31 < DMNB36 OK LOGOUT completed

======

----------------------------------------------------------------------

>Comment By: Joe (warehauz)
Date: 2004-04-26 21:40

Message:
Logged In: YES 
user_id=914399

Ok, after posing to the mailing list & working with Tony,
this one is resolved ... this IMAP server was not properly
returning the new messages in the SEARCH command after the
APPEND.  The workaround is to issue a NOOP before the
subsequent SEARCH.

Around line 510 of sb_imapfilter.py, after the
self._check(response, 'store') line, add:

imap.noop()

This issue appears to only affect certain IMAP servers.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498104&aid=941596&group_id=61702



More information about the Spambayes-bugs mailing list