[Spambayes] Problem with imap under linux.-part 7

[Tony Meyer]

> Tony, you are right that
> when I use i.list(), as I should of, I get a long list of files. I
> have not a clue at this time why those files were chosen. For example,
> why '(\\NoInferiors \\UnMarked) "/" .vue/types/tools/DigitalClock',
> '(\\NoInferiors \\UnMarked) "/" .profile' or  '(\\NoInferiors
> \\UnMarked) "/" .bashrc' were returned? These are not mail files, nor
> are they in my local user Mail directory.

This is something to do with the configuration of your mail server, and
that's all I know.  All that the i.list() command does is send 'command_id
LIST * ""' to the IMAP server - if you telnetted to it and send the same
command (after logging in), you'd get the same response.

> But you comment above seem to indicate that the
> mail sever should be running IMAP as well as the client running
> sb_imapfilter.py. You diagrams seem to indicate that sb_server.py and
> sb_imapfilter.py are the only IMAP and POP3 servers you need.

sb_server is a POP3 proxy, not a server* (i.e. traffic to/from a POP3 server
goes _through_ it).  sb_imapfilter is neither a proxy nor a server (i.e. it
connects directly to an IMAP4 server and does some filtering).

> Do you also need to have these servers running on the mail server.

You need to have an existing mail server, yes.

> So are
> these sb_* programs really pop3 and imap clients not servers?

sb_imapfilter is a form of IMAP client, yes.  sb_server is neither a client
nor a server, but something that sits in between the two.

[third message, explaining that accessing mail folders without sb_imapfilter
works]
> This is at variance with the suggestion that the spambayes and or 
> python imap access is not being able to access the correct mail folders
> because imap on the mail server is misconfigured.

I did say that this would work.  It's only getting the list of available
folders that is problematic.  Your mail client will most likely give you the
same very large list here, too, unless it's only asking for a subset of
folders.

It's nothing to do with Python, and nothing to do with SpamBayes.  The way
to demonstrate this is to telnet to the mail server, log in, and get the
list.

E.g.

$ telnet
telnet> o mail.example.com 143
Trying xxx.xxx.xxx.xxx...
Connected to mail.example.com.
Escape character is '^]'.
* OK [CAPABILITY IMAP4REV1 LOGIN-REFERRALS AUTH=LOGIN] mail.example.com
IMAP4rev1 2003.339-cpanel at Mon, 9 Aug 2004 15:51:56 +1200 (NZST)
0 login username password
0 OK [CAPABILITY IMAP4REV1 IDLE NAMESPACE MAILBOX-REFERRALS BINARY UNSELECT
SCAN SORT THREAD=REFERENCES THREAD=ORDEREDSUBJECT MULTIAPPEND] User username
authenticated
1 list "" *
* LIST (\NoInferiors \UnMarked) "/" ham_to_train
* LIST (\NoInferiors \UnMarked) "/" spam_to_train
* LIST (\NoInferiors \UnMarked) "/" spam
* LIST (\NoInferiors \UnMarked) "/" unsure
* LIST (\NoInferiors) NIL INBOX
1 OK LIST completed
2 logout
* BYE mail.example.com IMAP4rev1 server terminating connection
2 OK LOGOUT completed
Connection closed by foreign host.

If you try this, you'll get the same massive list after typing '1 list ""
*'.

> Is there anyone out there using a Linux based
> Spambayes system that can shed light on the problem?

I assure you that this problem is not platform specific at all.

[forth message]
> All that was needed is to identify the folders that
> IMAP needed to look at in the configuration file. The
> default INBOX for example would not cut it on a Linux system.

I gather that this means that you found the misconfigured bit in the IMAP
server.  It's still not a Linux-specific problem :)

[second message]
> I misspoke above. What I meant to say is sb_server.py and 
> sb_imapfilter.py are POP3 and IMAP proxy servers.
> They act as clients to the respective servers on the mail
> server but act as servers to the users of the local machine.

This is true for sb_server, yes.  sb_imapfilter is not a proxy, however - a
proxy doesn't work as well with IMAP (because the mail stays on the
machine), and, in many ways, a filter is a more desirable situation (e.g.
you can have spam moved to a separate folder, and then only periodically
download that folder's mail; you can't do this with POP3).  IAC, back when
the decision to write a script to deal with IMAP was made, the users wanted
a filter more than a proxy, so that was what they got.

> On our Linux lab machines we have nearly 2,000 potential
> users on each client machine and their use can be simultaneous
> on each machine. Now any of the 2,000 users can decide
> independently that they want to use POP3 or IMAP. This together
> with the suggestion that each user should use a separate server
> makes things seem very messy and made it hard to believe that
> to run spambayes we could have lets say 2 users using sb_server.py 
> and 3 users using sb_imapfilter.py with each having their own
> proxy server running on the local machine. Is this correct?

There would be two sb_server processes acting as POP3 proxies, and three
sb_imapfilter processes regularly connecting to the IMAP4 server, yes.

(Note that neither of these are particularly expensive in terms of memory or
processor time.  sb_server does need at least two local ports per instance,
though).

It's not all that clear (to me) whether you are after server side filtering,
or client side.

If you want server side filtering (i.e. it's all done before the user sees
it; there's a shared database; it's all centrally managed), then neither
sb_server nor sb_imapfilter are designed for this.  You can fit SpamBayes in
by using various tools (eg. the stuff on the server_side webpage), although
you might be better off with a different solution (like SpamAssassin).

If you want client side filtering (i.e. the user has all the control, they
get all of the mail, spam or not), then sb_server/sb_imapfilter are designed
for this.  For the most part, the user needs to be doing the work to manage
the system (that's part of pretty much any client side solution).  All that
you would really need to do is provide access to the sb_server/sb_imapfilter
scripts (and python) and some documentation.

If you could find someone to write it, it sounds like you want slightly
different scripts:

  * A version of sb_server that runs on the local machine and uses the USER
value (i.e. the username) and mail server name to determine which
configuration file to use.  It could then run independent of which user is
logged in, setup to proxy all the allowed POP3 servers, and act
appropriately depending which user connects.  This could be slow, however,
since it would have to read in the config file and databases each time a
connection to the mail server is made.  The web interface would have to be
similarly altered (extending the authentication code that's there, perhaps).

  * A script that runs sb_imapfilter on the local machine (i.e. loads the
user's config file, which specifies their database & folder setup) for all
the users that are logged in.

[fifth message, with update for server_side page]

Thanks for that.  I'll update the page shortly.

=Tony Meyer

* The "server" in the name comes from the fact that it's serving the web
interface, the POP3 proxy, and so on.  Eventually it may serve other things
as well.  Once upon a time the script was called pop3proxy.py.

---
Please always include the list (spambayes at python.org) in your replies
(reply-all), and please don't send me personal mail about SpamBayes. This
way, you get everyone's help, and avoid a lack of replies when I'm busy.