[spambayes-bugs] [ spambayes-Bugs-1023797 ] Imapfilter fails: 'Cannot find saved message'

Thu Dec 22 02:55:42 CET 2005

Bugs item #1023797, was opened at 2004-09-07 11:18
Message generated for change (Comment added) made by aikiaboy
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1023797&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: imapfilter
Group: Source code - CVS
Status: Open
Resolution: None
Priority: 2
Submitted By: Jonathan M. Gilligan (jmgilligan)
Assigned to: Tony Meyer (anadelonbrin)
Summary: Imapfilter fails: 'Cannot find saved message'

Initial Comment:
sb_imapfilter.py fails intermittently, but frequently, 
with the error:

__main__.BadIMAPResponseError: The 
command 'Cannot find saved message' failed to give an 
OK response.

Rerunning the filter produces this error, but different 
amounts of processing occur before I get the error. For 
what it's worth, my email client (mulberry) reports that 
it frequently spends as much as 90 seconds waiting for 
a response from the server, so perhaps this is a timeout 
problem.

I'm running the latest spambayes checked out from 
CVS using python 2.3.4 under Windows 2000 (Python 
2.3.4 (#53, Aug 23 2004, 16:24:32) [MSC v.1310 32 
bit (Intel)] on win32).

The server welcome is:
'* OK imap1 Cyrus IMAP4 v2.0.16 server ready'

The server capabilties is:
('IMAP4', 'IMAP4REV1', 'ACL', 'QUOTA', 'LITERAL+', 'NA
MESPACE', 'UIDPLUS', 'ID',
 'NO_ATOMIC_RENAME', 'UNSELECT', 'MULTIAPPEND', 'S
ORT', 'THREAD=ORDEREDSUBJECT',
 'THREAD=REFERENCES', 'IDLE', 'STARTTLS', 'AUTH=PL
AIN', 'X-NETSCAPE')

Attached is the output from sb_imapfilter -v -t -c -i 4 
where the exception occurs.

----------------------------------------------------------------------

Comment By: aikiaboy (aikiaboy)
Date: 2005-12-21 19:55

Message:
Logged In: YES 
user_id=1252668

It's probably not a good idea to use RECENT to check for 
messages because race conditions are possible with other 
IMAP clients, e.g. thunderbird and sb_imapfilter.py.

 According to the IMAP RFC: 

If multiple connections have the same mailbox selected 
simultaneously, it is undefined which of these connections 
will see newly-arrives messages with \Recent set and which 
will see it without \Recent set.

----------------------------------------------------------------------

Comment By: Jonathan M. Gilligan (jmgilligan)
Date: 2004-10-25 11:29

Message:
Logged In: YES 
user_id=11595

Following up after a few weeks of the latest version: I'm still 
getting occasional failures---probably some peculiarity of the 
Cyrus IMAP server I'm using. The failures are about once 
every day or two with about 8-10 hours of continual 
running "sb_imapfilter -v -t -c -l 10".

This is acceptable, and is a huge improvement over the state 
when I submitted this bug, where sb_imapfilter would crash 
several times per hour, but it would be nice if it were 
possible to fix it completely. Definitely a low-priority item, 
though, since it only affects the usability of the IMAP filter 
slightly.

----------------------------------------------------------------------

Comment By: Jonathan M. Gilligan (jmgilligan)
Date: 2004-10-13 16:16

Message:
Logged In: YES 
user_id=11595

I have been testing this for over a week with very few 
problems. I have gone from several failures per hour to less 
than one failure per day on average.

sjoerd's fix (rev. 1.40) seems to have made a lot of 
difference. Things are currently stable enough for me to be 
happy downgrading the severity of the bug. It's not quite 
perfectly fixed, but it's good enough for me to use regularly 
on my production system. 

I'll leave it up to Tony to decide whether this is sufficient to 
declare the bug closed.

----------------------------------------------------------------------

Comment By: Jonathan M. Gilligan (jmgilligan)
Date: 2004-10-04 16:01

Message:
Logged In: YES 
user_id=11595

Revision 1.40 (by sjoerd) to manage illegal values in the 
Message-Id, seems to fix this. I am currently testing and will 
report back on whether this really solves the problem.

----------------------------------------------------------------------

Comment By: Jonathan M. Gilligan (jmgilligan)
Date: 2004-09-30 10:41

Message:
Logged In: YES 
user_id=11595

I have increased the range in line 548 to 10000 or even 
100000, and it doesn't solve the problem.

I even inserted an increasing time.sleep():

for i in xrange(11):
   ....
   time.sleep(1 << i)

and I still get crashes, so just increasing the number of 
noops or waiting longer doesn't seem to solve the problem.

----------------------------------------------------------------------

Comment By: Tony Meyer (anadelonbrin)
Date: 2004-09-29 22:12

Message:
Logged In: YES 
user_id=552329

(Apologies for not getting to this yesterday)

Could you try making a change to your copy of
sb_imapfitler.py for me?

Line 548 is "        for i in xrange(100):".  Could you try
increasing the 100 to 10000 and seeing if that solves the
problem?  (It may run for a long time before it finally
quits, if it doesn't fix it - but it's just no-op'ing, so it
doesn't hurt).

I suspect it is related to a timeout sort of problem - maybe
the server is slow and so it's taking a long time to
register that the new message is there.  I guessed that 100
no-ops would be enough, but perhaps it isn't.

----------------------------------------------------------------------

Comment By: Tony Meyer (anadelonbrin)
Date: 2004-09-27 02:24

Message:
Logged In: YES 
user_id=552329

Apologies for the delay in getting to this (I've been overseas).

This code is quite significantly changed (for the better, in
theory) than the 1.0 sb_imapfilter, so hasn't been as widely
tested.  Your server isn't giving up the EXISTS response
that's expected or in the expected way, so something's
falling apart.

I'll try and find time to have a proper look at this
tomorrow.  Note that you ought to be able to fall back to
the 1.0 imapfilter if necessary.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1023797&group_id=61702