Mailman 3 February 2003 - SpamBayes

Outlook plugin - training
by Moore, Paul Dec. 3, 2003

Dec. 3, 2003

When the Outlook plugin filters mails, it classifies them as either spam or potential spam, and can put them in appropriate folders. In the spam/potential spam folders, there is a "Recover from Spam" button available, and in other folders there is a "Delete as spam" button. These buttons add the message to the training database as well as taking the appropriate action. One thing I don't see, however, is a means of confirming the classifier's decisions as correct. A "yes, that is spam" button for the spam folder, and a "yes, that's ham" button in non-spam folders. As I'm starting from a very small message base, I worry that correct classifications are still somewhat based on "luck", and training based on correct decisions would help to increase both my and the classifier's confidence level. I can do this by regular retraining, but that has 2 disadvantages: it's much clumsier than simply clicking on a "clever boy!" button, and it relies on me not deleting messages until I do a training run. Much of the ham I get is "read and forget", so I'd rather delete immediately. When I get a chance to dive into the code, I'll see how hard this would be to implement. Paul.

7 8

RE: [Spambayes] Outlook plugin - training
by Piers Haken Dec. 3, 2003

Dec. 3, 2003

I don't believe you need this. I think that the classifier automatically trains on messages as they arrive (or at least on messages that it's sure about). You only need to retrain if it has made a mistake, or if it's unsure. Piers. > -----Original Message----- > From: Moore, Paul [mailto:Paul.Moore@atosorigin.com] > Sent: Wednesday, November 06, 2002 2:09 AM > To: Spambayes (E-mail) > Subject: [Spambayes] Outlook plugin - training > > > When the Outlook plugin filters mails, it classifies them as > either spam or potential spam, and can put them in > appropriate folders. > > In the spam/potential spam folders, there is a "Recover from > Spam" button available, and in other folders there is a > "Delete as spam" button. These buttons add the message to the > training database as well as taking the appropriate action. > > One thing I don't see, however, is a means of confirming the > classifier's decisions as correct. A "yes, that is spam" > button for the spam folder, and a "yes, that's ham" button in > non-spam folders. > > As I'm starting from a very small message base, I worry that > correct classifications are still somewhat based on "luck", > and training based on correct decisions would help to > increase both my and the classifier's confidence level. > > I can do this by regular retraining, but that has 2 > disadvantages: it's much clumsier than simply clicking on a > "clever boy!" button, and it relies on me not deleting > messages until I do a training run. Much of the ham I get is > "read and forget", so I'd rather delete immediately. > > When I get a chance to dive into the code, I'll see how hard > this would be to implement. > > Paul. > > _______________________________________________ > Spambayes mailing list > Spambayes(a)python.org http://mail.python.org/mailman/listinfo/spambayes >

10 18

Advantages of S?
by Gary Robinson April 26, 2003

April 26, 2003

At present, it appears that S may perform better than relatively untweaked graham (as I guess bogofilter is using), but about the same as tweaked graham as spambayes is using, but has the advantage of showing a spread of results. As I see it there are two possible advantages to the spread: 1) emails in the middle could be classified as "you should manually check these", or 2) the user could adjust for the level of sensitivity he wants (similar to what happens now in Entourage which is otherwise brain-dead as a spam filter). (1) is a attractive but one thing worries me: it may give people a false sense of security -- they may think they only have to look at the middle ones, but occasionally even extreme cases of seeming to be spam might be legit. On the other hand, on those rare occasions, the sender can always try again if he doesn't get a response! ;) Just a couple of thoughts. --Gary -- Gary Robinson CEO Transpose, LLC grobinson(a)transpose.com 207-942-3463 http://www.emergentmusic.com http://radio.weblogs.com/0101454

8 19

Use of email package
by Tim Stone - Four Stones Expressions March 17, 2003

March 17, 2003

It appears from the bug report that Jacques has submitted, and from other errors I've seen in the pop3proxy lately, that the use of the email package is causing us some occasional trouble. The email package makes assumptions about the well-formedness of mail, and throws exceptions if those assumptions do not appear to apply to the mail being dealt with. I feel that this represents a fairly serious weakness in our solution, because spammers can exploit those assumptions to break the filter, thus discrediting it and making it harder for users to see their mail, and at the same time getting their spam through when users really do see their mail. We've got to either seriously harden our code so it knows what to do when the email package raises an exception, or consider not using the email package. I think I'll be reworking pop3proxy so that it no longer uses the email package for anything. The Corpus stuff currently has most (all?) the function that is needed by pop3proxy anyway. c'est moi - TimS http://www.fourstonesExpressions.com http://wecanstopspam.org

5 6

Other pop3proxy options
by Zander March 17, 2003

March 17, 2003

I would like to extend the options for how disposition is identified by the pop3proxy implementation. In particular, I would like the option of A. X-Spambayes-Classification: <disposition> as now B. To: XXXXX <disposition> as is in CVS now C. Subject line munging to append <disposition> Is there any reason that was not included? (beside the obvious potential for a spammer to slip in a workaround) I use Outlook Express, and obviously can't use the arbitrary header technique - and am most interested in adding a [***SPAM***] header so that I can correctly bucketize those messages - but leave [***UNSURE***] in my primary box, and not molest ham messages at all. Is there any reason not to do this? Would you accept it if I did? Is there any reason why you aren't using the email module Parser API to crack the headers? I have found a certain number of messages are not parsed correctly by the re that you are using. They show up as From: (none) Subj: (none) in the UI - but I haven't determined why just yet (though I can see that some part of the message is getting stuck with the header by your re.split(r'\n\r?\n', messageText, 1) expression. - Z

3 2

Adding a message database
by Mark Hammond March 6, 2003

March 6, 2003

I've been harping on about this for a while, and recently started seeing other people with the same need. If I glanced the checkin message correctly, TimS also wants one for his Notes work. Currently, core spambayes maintains a database of wordinfos. I would like spambayes to assist in managing a database of message_ids, mapped to how they were previously trained. While spambayes does not need any such concept to perform basic scoring, it seems that many applications using spambayes do. I see 2 strategies: 1) sub-class Classifier, adding this capability. This keeps it out of the core, but then makes it hard to use the existing sub-classes - eg, who does "DBStorage" derive from? We will either need to multiply out all the base classes, or use mixins. 2) Add the basic support to classifier, but in a non-intrusive way, allowing it to be left unused by an application. I believe that modifying Classifier to use a "Message object" is too intrusive. Specifically, for (2), I would change learn to: def learn(self, wordstream, is_spam, msg_id = None): ... self._add_msg(wordstream, is_spam) if msg_id is not None: self._add_msgid(msg_id, is_spam) Thus, if msg_id is never passed to learn (as no current application will), the new "_add_msgid()" function will never be called. Similarly for unlearn. _add_msgid() and _remove_msgid() would maintain a new dictionary-like object for the database, but it would initially be set to None, and demand-loaded first time it is actually needed, and saved only when non-None. The storage.py related sub-classes then get to implement a DB behind this. Pickles can save to a discrete file, while bsddb can use a multiple-databases-in-one-file approach. For existing applications, no attempt is ever made to load a "message database". Only when the app starts passing message IDs will the database be used, so this is completely up to the author of the app. It should also not break any existing clients or code. Comments? Mark.

6 18

OT: FYI: Microsoft virus email
by David LeBlanc Feb. 28, 2003

Feb. 28, 2003

I don't think this is really from Microsoft! I received this HTML email with the following headers: Return-Path: <has(a)inspire.net.nz> Delivered-To: alias-oznet-whisper(a)oz.net Received: (qmail 22407 invoked from network); 27 Feb 2003 02:03:05 -0000 Received: from 203-114-128-8.inspire.net.nz (HELO mail1.inspire.net.nz) (203.114.128.8) by smtp4.sea.theriver.com with SMTP; 27 Feb 2003 02:03:05 -0000 Received: from localhost (localhost.localdomain [127.0.0.1]) by mail1.inspire.net.nz (Postfix) with ESMTP id AC79B1ECD4E; Thu, 27 Feb 2003 15:02:50 +1300 (NZDT) Received: from mail1.inspire.net.nz ([127.0.0.1]) by localhost (mail1 [127.0.0.1:10024]) (amavisd-new) with ESMTP id 25347-09; Thu, 27 Feb 2003 15:02:45 +1300 (NZDT) Received: from DdgHKS (203-114-134-163.inspire.net.nz [203.114.134.163]) by mail1.inspire.net.nz (Postfix) with SMTP id 982221ECD0E; Thu, 27 Feb 2003 15:01:27 +1300 (NZDT) From: "Microsoft Internet Security Section" <tibhro-116500(a)OxiZlcPnb.com> To: "Microsoft User" <> SUBJECT: Microsoft Security Update Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="vaqcugFTSNuqXUJnWHrD" Message-Id: <20030227020127.982221ECD0E(a)mail1.inspire.net.nz> Date: Thu, 27 Feb 2003 15:01:27 +1300 (NZDT) X-Virus-Scanned: by amavisd-new -----Original Message----- From: Microsoft Internet Security Section [mailto:tibhro-116500@OxiZlcPnb.com] Sent: Wednesday, February 26, 2003 18:01 To: Microsoft User Subject: Microsoft Security Update Microsoft User this is the latest version of security update, the "February 2003, Cumulative Patch" update which eliminates all known security vulnerabilities affecting Internet Explorer, Outlook and Outlook Express as well as five newly discovered vulnerabilities. Install now to protect your computer from these vulnerabilities, the most serious of which could allow an attacker to run executable on your system. This update includes the functionality of all previously released patches. System requirements Win 9x/Me/2000/NT/XP This update applies to Microsoft Internet Explorer, version 4.01 and later Microsoft Outlook, version 8.00 and later Microsoft Outlook Express, version 4.01 and later Recommendation Customers should install the patch at the earliest opportunity. How to install Run attached file. Click Yes on displayed dialog box. How to use You don't need to do anything after installing this item. Microsoft Product Support Services and Knowledge Base articles can be found on the Microsoft Technical Support web site. For security-related information about Microsoft products, please visit the Microsoft Security Advisor web site, or Contact us. Please do not reply to this message. It was sent from an unmonitored e-mail address and we are unable to respond to any replies. Thank you for using Microsoft products. With friendly greetings, Microsoft Internet Security Section ---------------------------------------------------------------------------- ---- )2003 Microsoft Corporation. All rights reserved. The names of the actual companies and products mentioned herein may be the trademarks of their respective owners. ________________________________________________ I didn't think anyone would like the attachment ;-) David LeBlanc Seattle, WA USA

3 2

[Spambayes] [ spambayes-Bugs-695142 ] Email does not render subject in the "Review" Page
by SourceForge.net Feb. 28, 2003

Feb. 28, 2003

Bugs item #695142, was opened at 2003-02-28 10:40 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=498103&aid=695142&group_i… Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: David Shaw (dshaw) >Assigned to: Tim Stone (timstone4) >Summary: Email does not render subject in the "Review" Page Initial Comment: I received the attached email. When I go to the "review" web page of pop3proxy.py, all it shows is: Messages classified as Unsure: From: (none) (none) It acts as though the message has no "from" or "subject", even though they exist. The user is not given any way to classify this message other than to click on the first "(none)" and read the raw message to determine its contents. I will attach the message below. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=498103&aid=695142&group_i…

1 0

[Spambayes] [ spambayes-Bugs-695187 ] mboxtrain.py and dbExpImp.py not Python 2.2 compatible
by SourceForge.net Feb. 28, 2003

Feb. 28, 2003

Bugs item #695187, was opened at 2003-02-28 11:45 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=498103&aid=695187&group_i… Category: None Group: None >Status: Closed Resolution: None Priority: 5 Submitted By: David Shaw (dshaw) Assigned to: Tim Stone (timstone4) Summary: mboxtrain.py and dbExpImp.py not Python 2.2 compatible Initial Comment: Both mboxtrain.py and dbExpImp.py lack the following compatibility statements that let them work in Python 2.2. Other files seem to have them, so these two should as well. try: True, False except NameError: # Maintain compatibility with Python 2.2 True, False = 1, 0 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=498103&aid=695187&group_i…

1 0

[Spambayes] [ spambayes-Bugs-695187 ] mboxtrain.py and dbExpImp.py not Python 2.2 compatible
by SourceForge.net Feb. 28, 2003

Feb. 28, 2003

Bugs item #695187, was opened at 2003-02-28 11:45 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=498103&aid=695187&group_i… Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: David Shaw (dshaw) >Assigned to: Tim Stone (timstone4) Summary: mboxtrain.py and dbExpImp.py not Python 2.2 compatible Initial Comment: Both mboxtrain.py and dbExpImp.py lack the following compatibility statements that let them work in Python 2.2. Other files seem to have them, so these two should as well. try: True, False except NameError: # Maintain compatibility with Python 2.2 True, False = 1, 0 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=498103&aid=695187&group_i…

1 0