Inconsistencies in classification between my mail reader and the web interface - being flooded by spam
Backdrop: I've been using Spambayes for a couple of years, and have had it running successfully on many computers, both Windows and Linux. This message concerns installations on Linux boxes. In the past few months, I've been seeing a steady deterioration of Spambayes' performance, and it seemed like I was seeing messages in the Web Interface Review page that were being classified as spam, yet, in my mail reader, they were winding up in the Inbox or my unsure folder. Tonight, I decided to take the time and confirm that. Below are two headers from the same message, one is a cut/paste from my Kmail reader window, and the other is a cut/paste from my Spambayes Web Interface review page. There's several things that show it's the exact same message. Note the ESMTPid which are both k8S0Y57Q009225, and the X-Spambayes-MailId which are both 1159403792. YET, one is classified as spam and the other is classified as unsure. How could this be happenning? Sometime in the past year, someone incorporated code in Kmail that allows it to detect the presence of Spambayes on the machine in a wizard interface, and it configures a few filters for you if you choose to allow that. The filters are fairly innocuous, and seem to do what they're supposed to. But, I'm wondering if they're the problem. The filter merely examines the spambayes classification line in the header and sends spam to the trash bin, and unsure to a folder of your choosing, seems right to me. Then, it configures a couple of buttons that appear on the toolbar, and those are linked to manual filtering actions: "sb_filter.py -s" for the spam button and "sb_filter.py -g" for the not spam button. If the Spambayes proxy-server is processing the message, how could one version appear in my mail reader's viewing pane, and a different one appear in the web interface??? This is very strange. Thanking anyone, in advance, who has the time to look at this. Kmail viewing pane message header Return-Path: <4seasons@krankination.com> Received: from pam2.krankination.com ([72.11.149.121]) by jefferson.patriot.net (8.13.4/8.13.4/Debian-3sarge1) with ESMTP id k8S0Y57Q009225 for <claude_jones@levitjames.com>; Wed, 27 Sep 2006 20:34:10 -0400 X-Mailer: Mcamp:int6-1.1-1110-Sq X-Filter_AZ: WTJ4aGRXUmxYMnB2Ym1WelFHeGxkbWwwYW1GdFpYTXVZMjl0Cg== X-Filter_BZ: TVRNekxraE5UbDgwYzJWaGMyOXVjM1I0ZERZeE15NUpVazlPVUU5U1ZESXROVEkzCg== X-Filter_CZ: UVZWSE1EWnZjQT09Cg== MIME-Version: 1.0 Date: 27 Sep 2006 20:33:44 -0400 Message-ID: <WTJ4aGRXUmxYMnB2Ym1WelFHeGxkbWwwYW1GdFpYTXVZMjl0Cg==@krankination.com> Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7Bit From: International Wines <4seasons@krankination.com> To: claude_jones@levitjames.com Subject: 12 Bottles of Fine Wine and exclusive gift only 59.88 US dollars Status: R X-UID: 97982 Content-Length: 4062 X-Keywords: X-Spambayes-MailId: 1159403792 X-Spambayes-Classification: unsure; 0.29 X-Status: NC X-KMail-EncryptionState: X-KMail-SignatureState: X-KMail-MDN-Sent: Spambayes web interface view message interface header of same message: Return-Path: <4seasons@krankination.com> Received: from pam2.krankination.com ([72.11.149.121]) by jefferson.patriot.net (8.13.4/8.13.4/Debian-3sarge1) with ESMTP id k8S0Y57Q009225 for <claude_jones@levitjames.com>; Wed, 27 Sep 2006 20:34:10 -0400 X-Mailer: Mcamp:int6-1.1-1110-Sq X-Filter_AZ: WTJ4aGRXUmxYMnB2Ym1WelFHeGxkbWwwYW1GdFpYTXVZMjl0Cg== X-Filter_BZ: TVRNekxraE5UbDgwYzJWaGMyOXVjM1I0ZERZeE15NUpVazlPVUU5U1ZESXROVEkzCg== X-Filter_CZ: UVZWSE1EWnZjQT09Cg== MIME-Version: 1.0 Date: 27 Sep 2006 20:33:44 -0400 Message-ID: <WTJ4aGRXUmxYMnB2Ym1WelFHeGxkbWwwYW1GdFpYTXVZMjl0Cg==@krankination.com> Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7Bit From: International Wines <4seasons@krankination.com> To: claude_jones@levitjames.com Subject: 12 Bottles of Fine Wine and exclusive gift only 59.88 US dollars Status: O X-UID: 97982 Content-Length: 4062 X-Keywords: X-Spambayes-Classification: spam X-Spambayes-MailId: 1159403792 -- Claude Jones Brunswick, MD, USA
On Wednesday 27 September 2006 22:10, Claude Jones wrote:
Sometime in the past year, someone incorporated code in Kmail that allows it to detect the presence of Spambayes on the machine in a wizard interface, and it configures a few filters for you if you choose to allow that. The filters are fairly innocuous, and seem to do what they're supposed to. But, I'm wondering if they're the problem. The filter merely examines the spambayes classification line in the header and sends spam to the trash bin, and unsure to a folder of your choosing, seems right to me. Then, it configures a couple of buttons that appear on the toolbar, and those are linked to manual filtering actions: "sb_filter.py -s" for the spam button and "sb_filter.py -g" for the not spam button.
If the Spambayes proxy-server is processing the message, how could one version appear in my mail reader's viewing pane, and a different one appear in the web interface??? This is very strange.
I think I've found the problem, though the cause is yet not clear, so I post this in case others encounter this issue. Kmail has a wizard for configuring anti-spam filtering. If you click on it, the first thing it does is look for several known anti-spam programs running on your system. If it finds any, it then produces a list. Highlighting any listed program activates a routine in which it then asks where you want to direct detected spam, and probable spam in some cases, and a final click writes a few rules into the filter stack. One of those rules is to pipe all messages through Spambayes to be classified. However, on my system as configured, Spambayes is started up at boot time, and is configured to run checks on all the mail as it comes in by piping it through the proxy server. So, I do a mail check; incoming mail goes through the proxy server and then gets processed by Kmail; Kmail compares the mail against the filter stack, sending known quantities to various folders, and, if the mail message hasn't been processed by any filters, one of the last filters is a pipe to run a Spambayes check. That's the source of the reason one message gets processed, and can appear in two different places with different Spambayes header classifications. It's literally getting two Spambayes checks. In my cited example, the incoming message went through the proxy server and was classified as spam. Then, it was processed by Kmail, and didn't meet any of the filter criteria so it was then piped through a Spambayes check for A SECOND TIME. Now, the only thing that is confusing to me is why there was one analysis that concluded it was spam, and a second that concluded it was unsure... I may be on to the answer, partially. Each time I did this, I was starting over my database anew by deleting three files, which was not quite what the faq instructions stated, by the way - they were .hammie.db hammie.db and spambayes.messageinfo.db Note the two versions of hammie.db - I believe the one without the dot at the beginnning was set up the Kmail wizard - that would mean that the incoming messages were being processed against two different databases! - I guess that would cause the behavior I experienced. I note that now, I've removed the pipe in my Kmail filter stack that sends the message through Spambayes a second time, and now, no hammie.db file has been created, only the .hammie.db --- Well, I had no responses to my first post, but, maybe this will be helpful to someone. If I don't post further followup, assume that the issue is solved. I only had one false negative in my inbox this morning, and about 11 unsures. That's a huge improvement over what I had been getting, just after one evening's training of a new database. -- Claude Jones Brunswick, MD, USA
participants (1)
-
Claude Jones