[Spambayes] Errors when setting up IMAP folders to filter and trainusing the web interface

[Joshua L Ross]

On Mon, 2004-03-08 at 14:04, Tony Meyer wrote:
> [Sorry for the delay in replying; been very busy lately.]

That is no problem whatsoever.  I am so used to the spam that it doesn't
bother me a few more days of it.  Also, I can switch my mail to pop if
it were so bad that it was necessary.  Truly, I raised the problem
because I couldn't find another example of it and I want to help you all
resolve any problems to make spambayes better.  With that said, I
suspect this might be a problem with the way my company's imap server is
configured, or something I am doing incorrectly.  So I am trying to
investigate possible mistakes on my end and sending messages to you all.


> 
> > 	I am attempting to use the command "sb_imapfilter.py 
> > -b" to start the web configuration for IMAP.  When I get into 
> > the configuration webpage I am unable to "configure folders 
> > to filter" or "Configure folders to train".  When I click on 
> > either of these links on the web configuration screen I 
> > receive this error message:
> > 
> [...]
> >   File "./sb_imapfilter.py", line 285, in folder_list
> >     m = r.search(fol)
> > 
> > TypeError: expected string or buffer
> > 
> > I ran the "sb_imapfilter.py -b" with debugging "-i 4 -v" and 
> > I received this output which I have shortened.  It apparently 
> > lists every file in my home directory in this debug
> 
> This is weird.  It's meant to list the directories available on the IMAP
> server.  I gather you're running a local IMAP server?  I guess maybe
> whatever server it is has decided that it'll return any files in that
> directory.
> 
> The IMAP server is certainly doing one thing wrong.  The folder names must
> (according to the RFC) be either a literal (something like {12} and then 12
> characters), a single character, or a *quoted* string.  This *might* be the
> problem; I'm not certain.
> 
> >   22:41.50 < * LIST (\NoInferiors \Marked) "/" 1
> >   22:41.50 < * LIST (\NoInferiors \Marked) "/" {1}
> >   22:41.50 read literal size 1
> 
> This one is odd.  Only one character, and for some reason it *is* a literal.
> 

It WAS a file named "/".  I actually accidentally saved a file in 'vi'
with that filename.  I removed a bunch of junk from my /trout/home/josh/
folder now, including that file.



> > I don't know much python but from what I can tell, even
> > if you set the fitler_folders and spam_folders in the
> > .spambayesrc file it still cannot find/use these using IMAP. 
> > I attempted to run a classification after setting the above
> > filter_folders option and received this response:
> >
> [...]
> > filter_folders=/trout/home/josh/INBOX
> > spam_folder=/trout/home/josh/mail/Newspammail
> > unsure_folder=/trout/home/josh/mail/Unsuremail
> [...]
> > Invalid response to select /trout/home/josh/mail/Newspammail:
> > ('NO', ["SELECT failed: Can't open /trout/home/josh/mail/Newspammail:
> > not a selectable mailbox"])
> >
> > This error message is coming from outside this script, and
> > I haven't been able to determine from where. 
> 
> It comes from imaplib (the Python standard module).  If you do:
> 
> python
> >>> import imaplib
> >>> i = imaplib.IMAP4("your.server.name")
> >>> i.login("username", "password")
> >>> i.select("/trout/home/josh/mail/Newspammail")


08:38:54|trout:~>python
Python 2.2.1 (#1, Jul  5 2002, 10:53:30) 
[GCC egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import imaplib
>>> i = imaplib.IMAP4("trout.compgen.com")
>>> i.login("josh", "********")
('OK', ['LOGIN completed'])
>>> i.select("/trout/home/josh/mail/Newspammail")
('NO', ["SELECT failed: Can't open /trout/home/josh/mail/Newspammail:
not a selectable mailbox"])
>>> i.select("/trout/home/josh/Inbox")
('NO', ["SELECT failed: Can't open mailbox /trout/home/josh/Inbox: no
such mailbox"])
>>> i.select("/trout/home/josh/INBOX")
('NO', ["SELECT failed: Can't open mailbox /trout/home/josh/INBOX: no
such mailbox"])

It doesn't appear any folder exists in my IMAP server?  It is odd, I
guess the fact that I could "subscribe" to 'Newspammail' in Evolution
gave me the idea that it was/is an Imap Folder.  I am apparently wrong.




> 
> What do you get?  Does that folder definitely exist on the IMAP server?
> 
> Could you maybe do another debug run trying to set the folders, but with
> -i9?  It's usually excessive, but it'll show exactly what imaplib is putting
> in the response.  As before, cut out chunks so that it's not overly huge.
> 
> =Tony Meyer
> 
> ---
> Please always include the list (spambayes at python.org) in your replies
> (reply-all), and please don't send me personal mail about SpamBayes. This
> way, you get everyone's help, and avoid a lack of replies when I'm busy.

HA!  something has changed on my end, probably the imap server?  I'm not
sure I am of much help now, because this time the web configuration
listed all the folders in my home directory.  Wait, I bet cleaning the
directory /trout/home/josh/ of junk and the "/" file name rectified the
web configuration problems in my previous mail.  Anyway, just in case,
here is the output:



josh at uxjosh:/usr/local/spambayes/scripts> ./sb_imapfilter.py -i9  -v -b
SpamBayes IMAP Filter Version 0.2 (February 2004)
and engine SpamBayes Engine Version 0.3 (January 2004).

Loading state from /home/josh/.hammiedb database
/home/josh/.hammiedb is an existing database, with 1 spam and 1 ham
Loading database /home/josh/.hammiedb... Done.
User interface url is http://localhost:8880/
  37:23.49 > DDPJ1 LOGIN josh "********"
  37:23.58 < DDPJ1 OK LOGIN completed
  37:23.58      matched r'(?P<tag>DDPJ\d+) (?P<type>[A-Z]+)
(?P<data>.*)' => ('DDPJ1', 'OK', 'LOGIN completed')
  37:23.58 > DDPJ2 LIST "" *
  37:23.67 < * LIST (\NoInferiors \Marked) "/" ld
  37:23.67      matched r'\* (?P<type>[A-Z-]+)( (?P<data>.*))?' =>
('LIST', ' (\\NoInferiors \\Marked) "/" ld', '(\\NoInferiors \\Marked)
"/" ld')
  37:23.67 untagged_responses[LIST] 0 += ["(\NoInferiors \Marked) "/"
ld"]
  37:23.67 < * LIST (\NoSelect) "/" bin
  37:23.67      matched r'\* (?P<type>[A-Z-]+)( (?P<data>.*))?' =>
('LIST', ' (\\NoSelect) "/" bin', '(\\NoSelect) "/" bin')
  37:23.67 untagged_responses[LIST] 1 += ["(\NoSelect) "/" bin"]
  37:23.67 < * LIST (\NoSelect) "/" etc
  37:23.67      matched r'\* (?P<type>[A-Z-]+)( (?P<data>.*))?' =>
('LIST', ' (\\NoSelect) "/" etc', '(\\NoSelect) "/" etc')
  37:23.67 untagged_responses[LIST] 2 += ["(\NoSelect) "/" etc"]
  37:23.67 < * LIST (\NoInferiors \Marked) "/" etc/fstab
  37:23.67      matched r'\* (?P<type>[A-Z-]+)( (?P<data>.*))?' =>
('LIST', ' (\\NoInferiors \\Marked) "/" etc/fstab', '(\\NoInferiors
\\Marked) "/" etc/fstab')
  37:23.67 untagged_responses[LIST] 3 += ["(\NoInferiors \Marked) "/"
etc/fstab"]
  37:23.67 < * LIST (\NoSelect) "/" tmp
  37:23.67      matched r'\* (?P<type>[A-Z-]+)( (?P<data>.*))?' =>
('LIST', ' (\\NoSelect) "/" tmp', '(\\NoSelect) "/" tmp')
  37:23.67 untagged_responses[LIST] 4 += ["(\NoSelect) "/" tmp"]
  37:23.67 < * LIST (\NoInferiors \Marked) "/" tmp/.created_by_xemacs
  37:23.67      matched r'\* (?P<type>[A-Z-]+)( (?P<data>.*))?' =>
('LIST', ' (\\NoInferiors \\Marked) "/" tmp/.created_by_xemacs',
'(\\NoInferiors \\Marked) "/" tmp/.created_by_xemacs')
  37:23.67 untagged_responses[LIST] 5 += ["(\NoInferiors \Marked) "/"
tmp/.created_by_xemacs"]
  37:23.67 < * LIST (\NoSelect) "/" .ssh
  37:23.67      matched r'\* (?P<type>[A-Z-]+)( (?P<data>.*))?' =>
('LIST', ' (\\NoSelect) "/" .ssh', '(\\NoSelect) "/" .ssh')



....
....
....
....

uto-save-list', '(\\NoSelect) "/" .emacs.d/auto-save-list')
  37:24.52 untagged_responses[LIST] 848 += ["(\NoSelect) "/"
.emacs.d/auto-save-list"]
  37:24.52 < * LIST (\NoInferiors \UnMarked) "/"
.emacs.d/auto-save-list/.saves-6857-bass.compgen.com~
  37:24.52      matched r'\* (?P<type>[A-Z-]+)( (?P<data>.*))?' =>
('LIST', ' (\\NoInferiors \\UnMarked) "/"
.emacs.d/auto-save-list/.saves-6857-bass.compgen.com~', '(\\NoInferiors
\\UnMarked) "/" .emacs.d/auto-save-list/.saves-6857-bass.compgen.com~')
  37:24.52 untagged_responses[LIST] 849 += ["(\NoInferiors \UnMarked)
"/" .emacs.d/auto-save-list/.saves-6857-bass.compgen.com~"]
  37:24.52 < * LIST (\NoInferiors \Marked) "/" SalaryReview.pps
  37:24.52      matched r'\* (?P<type>[A-Z-]+)( (?P<data>.*))?' =>
('LIST', ' (\\NoInferiors \\Marked) "/" SalaryReview.pps',
'(\\NoInferiors \\Marked) "/" SalaryReview.pps')
  37:24.52 untagged_responses[LIST] 850 += ["(\NoInferiors \Marked) "/"
SalaryReview.pps"]
  37:24.52 < * LIST (\NoInferiors \Marked) "/" maggianos.ps
  37:24.52      matched r'\* (?P<type>[A-Z-]+)( (?P<data>.*))?' =>
('LIST', ' (\\NoInferiors \\Marked) "/" maggianos.ps', '(\\NoInferiors
\\Marked) "/" maggianos.ps')
  37:24.52 untagged_responses[LIST] 851 += ["(\NoInferiors \Marked) "/"
maggianos.ps"]
  37:24.52 < * LIST (\NoInferiors \Marked) "/" .hist_josh
  37:24.52      matched r'\* (?P<type>[A-Z-]+)( (?P<data>.*))?' =>
('LIST', ' (\\NoInferiors \\Marked) "/" .hist_josh', '(\\NoInferiors
\\Marked) "/" .hist_josh')
  37:24.52 untagged_responses[LIST] 852 += ["(\NoInferiors \Marked) "/"
.hist_josh"]
  37:24.52 < * LIST (\NoInferiors) NIL INBOX
  37:24.52      matched r'\* (?P<type>[A-Z-]+)( (?P<data>.*))?' =>
('LIST', ' (\\NoInferiors) NIL INBOX', '(\\NoInferiors) NIL INBOX')
  37:24.52 untagged_responses[LIST] 853 += ["(\NoInferiors) NIL INBOX"]
  37:24.52 < DDPJ2 OK LIST completed
  37:24.52      matched r'(?P<tag>DDPJ\d+) (?P<type>[A-Z]+)
(?P<data>.*)' => ('DDPJ2', 'OK', 'LIST completed')
  37:24.52 untagged_responses[LIST] => ['(\\NoInferiors \\Marked) "/"
ld', '(\\NoSelect) "/" bin', '(\\NoSelect) "/" etc', '(\\NoInferiors
\\Marked) "/" etc/fstab', '(\\NoSelect) "/" tmp', '(\\NoInferiors
\\Marked) "/" tmp/.created_by_xemacs', '(\\NoSelect) "/" .ssh',
'(\\NoInferiors \\Marked) "/" .ssh/id.ssh.pub', '(\\NoInferiors
\\UnMarked) "/" .ssh/known_hosts2', '(\\NoInferiors \\Marked) "/"
.ssh/identity.pub', '(\\NoInferiors \\Marked) "/"
.ssh/authorized_keys~', '(\\NoInferiors \\UnMarked) "/" .ssh/identity',
'(\\NoInferiors \\UnMarked) "/" .ssh/id_dsa.pub', '(\\NoInferiors
\\UnMarked) "/" .ssh/id_rsa.pub', '(\\NoInferiors \\UnMarked) "/"
.ssh/id_dsa', '(\\NoInferiors \\UnMarked) "/" .ssh/id_rsa',
'(\\NoInferiors \\UnMarked) "/" .ssh/known_hosts', '(\\NoInferiors
\\Marked) "/" .ssh/random_seed', '(\\NoInferiors \\Marked) "/"
.ssh/authorized_keys', '(\\NoSelect) "/" .vim', '(\\NoSelect) "/"
.vim/doc', '(\\NoInferiors \\Marked) "/" .vim/doc/tags', '(\\NoInferiors
\\Marked) "/" .vim/doc/vimspell.txt', '(\\NoInferiors \\Marked) "/"
.vim/doc/FEATURES.txt', '(\\NoSelect) "/" .vim/ftplugin',
....
....
....
....

 \\Marked) "/"
.fullcircle/NetscapeNetscape6.00LinuxIntel2000110807/permdata.box',
'(\\NoInferiors \\Marked) "/"
.fullcircle/NetscapeNetscape6.00LinuxIntel2000110807/manifest.ini',
'(\\NoSelect) "/" .fullcircle/NetscapeNetscape6.21LinuxIntel2001112617',
'(\\NoInferiors \\Marked) "/"
.fullcircle/NetscapeNetscape6.21LinuxIntel2001112617/permdata.box',
'(\\NoInferiors \\Marked) "/"
.fullcircle/NetscapeNetscape6.21LinuxIntel2001112617/manifest.ini',
'(\\NoInferiors \\Marked) "/"
.fullcircle/NetscapeNetscape6.21LinuxIntel2001112617/Talkback.ini',
'(\\NoInferiors \\Marked) "/"
.fullcircle/NetscapeNetscape6.21LinuxIntel2001112617/info.db',
'(\\NoInferiors \\Marked) "/" cgi_ssh', '(\\NoSelect) "/"
Application_Kit_rev1', '(\\NoSelect) "/"
Application_Kit_rev1/Application_Kit_rev1', '(\\NoInferiors \\Marked)
"/" "Application_Kit_rev1/Application_Kit_rev1/Application
Worksheet_rev1.doc"', '(\\NoInferiors \\Marked) "/"
"Application_Kit_rev1/Application_Kit_rev1/Committment to Excellence
Form.doc"', '(\\NoInferiors \\Marked) "/"
"Application_Kit_rev1/Application_Kit_rev1/Interest Rate Agreement
Disclosure_rev1.doc"', '(\\NoInferiors \\Marked) "/"
"Application_Kit_rev1/Application_Kit_rev1/Authorization to Release
Information.doc"', '(\\NoInferiors \\Marked) "/"
"Application_Kit_rev1/Application_Kit_rev1/Application Checklist.doc"',
'(\\NoInferiors \\Marked) "/"
"Application_Kit_rev1/Application_Kit_rev1/Fee Disclosure_rev1.doc"',
'(\\NoInferiors \\Marked) "/"
"Application_Kit_rev1/Application_Kit_rev1/ABFA Disclosure_rev1.doc"',
'(\\NoInferiors \\Marked) "/"
"Application_Kit_rev1/Application_Kit_rev1/ABFA Tax
Disclosure_rev1.doc"', '(\\NoSelect) "/" .emacs.d', '(\\NoSelect) "/"
.emacs.d/auto-save-list', '(\\NoInferiors \\UnMarked) "/"
.emacs.d/auto-save-list/.saves-6857-bass.compgen.com~', '(\\NoInferiors
\\Marked) "/" SalaryReview.pps', '(\\NoInferiors \\Marked) "/"
maggianos.ps', '(\\NoInferiors \\Marked) "/" .hist_josh',
'(\\NoInferiors) NIL INBOX']




I am going to continue to configure this now that I am getting a
directory listing.  I will update this thread to see if that "/" file
might have been the whole problem.  Thanks for the help!

-Joshua-