Mailman 3 January 2003 - SpamBayes

Outlook plugin - training
by Moore, Paul Dec. 3, 2003

Dec. 3, 2003

When the Outlook plugin filters mails, it classifies them as either spam or potential spam, and can put them in appropriate folders. In the spam/potential spam folders, there is a "Recover from Spam" button available, and in other folders there is a "Delete as spam" button. These buttons add the message to the training database as well as taking the appropriate action. One thing I don't see, however, is a means of confirming the classifier's decisions as correct. A "yes, that is spam" button for the spam folder, and a "yes, that's ham" button in non-spam folders. As I'm starting from a very small message base, I worry that correct classifications are still somewhat based on "luck", and training based on correct decisions would help to increase both my and the classifier's confidence level. I can do this by regular retraining, but that has 2 disadvantages: it's much clumsier than simply clicking on a "clever boy!" button, and it relies on me not deleting messages until I do a training run. Much of the ham I get is "read and forget", so I'd rather delete immediately. When I get a chance to dive into the code, I'll see how hard this would be to implement. Paul.

7 8

RE: [Spambayes] Outlook plugin - training
by Piers Haken Dec. 3, 2003

Dec. 3, 2003

I don't believe you need this. I think that the classifier automatically trains on messages as they arrive (or at least on messages that it's sure about). You only need to retrain if it has made a mistake, or if it's unsure. Piers. > -----Original Message----- > From: Moore, Paul [mailto:Paul.Moore@atosorigin.com] > Sent: Wednesday, November 06, 2002 2:09 AM > To: Spambayes (E-mail) > Subject: [Spambayes] Outlook plugin - training > > > When the Outlook plugin filters mails, it classifies them as > either spam or potential spam, and can put them in > appropriate folders. > > In the spam/potential spam folders, there is a "Recover from > Spam" button available, and in other folders there is a > "Delete as spam" button. These buttons add the message to the > training database as well as taking the appropriate action. > > One thing I don't see, however, is a means of confirming the > classifier's decisions as correct. A "yes, that is spam" > button for the spam folder, and a "yes, that's ham" button in > non-spam folders. > > As I'm starting from a very small message base, I worry that > correct classifications are still somewhat based on "luck", > and training based on correct decisions would help to > increase both my and the classifier's confidence level. > > I can do this by regular retraining, but that has 2 > disadvantages: it's much clumsier than simply clicking on a > "clever boy!" button, and it relies on me not deleting > messages until I do a training run. Much of the ham I get is > "read and forget", so I'd rather delete immediately. > > When I get a chance to dive into the code, I'll see how hard > this would be to implement. > > Paul. > > _______________________________________________ > Spambayes mailing list > Spambayes(a)python.org http://mail.python.org/mailman/listinfo/spambayes >

10 18

Advantages of S?
by Gary Robinson April 26, 2003

April 26, 2003

At present, it appears that S may perform better than relatively untweaked graham (as I guess bogofilter is using), but about the same as tweaked graham as spambayes is using, but has the advantage of showing a spread of results. As I see it there are two possible advantages to the spread: 1) emails in the middle could be classified as "you should manually check these", or 2) the user could adjust for the level of sensitivity he wants (similar to what happens now in Entourage which is otherwise brain-dead as a spam filter). (1) is a attractive but one thing worries me: it may give people a false sense of security -- they may think they only have to look at the middle ones, but occasionally even extreme cases of seeming to be spam might be legit. On the other hand, on those rare occasions, the sender can always try again if he doesn't get a response! ;) Just a couple of thoughts. --Gary -- Gary Robinson CEO Transpose, LLC grobinson(a)transpose.com 207-942-3463 http://www.emergentmusic.com http://radio.weblogs.com/0101454

8 19

Alpha2 Pre-release
by Richie Hindle Feb. 13, 2003

Feb. 13, 2003

I've built an alpha2 source release of Spambayes. Before we put it up on the main web site, I'd feel a lot better if someone could smoke-test it for me - I may have made some horrible mistake that I'm too close to see... I've put it here: http://entrian.com/spambayes/spambayes-1.0a2-pre.zip http://entrian.com/spambayes/spambayes-1.0a2-pre.tar.gz For POP3 proxy users, this release should be GUI out of the box - install it, run pop3proxy.py, point your browser at the URL, go to the Config page and enter your POP3 server details, change your email client to point at the proxy, and you're away - messages are classfied and you can train through the web. For hammie users there's Neale's new muttrc and spambayes.el, and Skip's proxytee lets hammie users train through the web interface. Tim Stone's import/export script should make upgrading easy, for now and in the future. Assorted improvements to the tokeniser and classifier make spambayes even more accurate. What else has changed? We should do a proper release announcement - I don't keep up with the Outlook plug-in, so what's new there? Who've I offended by forgetting about their fantastic new feature? 8-) One question for those who know about these things: I originally built the release on Windows, but then realised that all the source files in both the zip and tar.gz archives had Windows line-endings. People installing and editing on unix would see '^M's all over the place (possibly, depending on their editor). Is there a distutils option I've missed to prevent this? Anyway, I rebuilt the archives on unix (thanks Neale!). -- Richie Hindle richie(a)entrian.com

4 4

Ximian Evolution
by Kevin Turner Feb. 6, 2003

Feb. 6, 2003

[apologies if you see this twice; sent it from the wrong address the first time and it stuck in the moderator queue.] A search of the list doesn't turn anything up, so I guess this is a new thread. Has anyone given any thought to how to use spambayes with the Evolution MUA without procmail or proxy? I just came up with something that sounds plausible; I haven't coded it yet. Current versions of Evolution 1.2 can filter messages based on the exit code of a process you pipe them to. Or you may be able to chain filters, first piping it through an action to add the spambayes header, then filtering on that.[1] Either way, filtering shouldn't be hard. For trailing, Evolution seems to lack a way to script it, but you could copy messages to folders[2] and run hammie on them at your leisure or via a cron job. (Does hammie move messages out of this mailbox or flag them as trained in some way so it doesn't double-train them?) It would be nice to avoid process-per-message overhead. Maybe bonobo holds some wonderful answer to that, I don't know. For now, I'm probably willing to not worry about it. Has anyone set this up? Can we add a section about it in the integration docs? That'd be super. Thanks, - Kevin [1] The filter-on-header method has the advantage of being easier to trace and debug, I only hesitate because I haven't convinced myself I know what order Evolution applies filters in and the conditions in which it chains them. [2] These could be in ~/evolution/local, or configured as a separate account of prococol "mbox" or "Maildir"; whichever option rankles evolution less, considering we may modify the contents of the mailbox without its knowledge. I'm guessing the "seperate account" option is probably better for that. -- The moon is new, 1.2% illuminated, 28.5 days old.

2 2

A question
by Francois Granger Feb. 1, 2003

Feb. 1, 2003

I recently received a spam properly classified as spam. I copy and past it content from Eudora in pop3proxy and click Classify. It give me a Spam probability: 0.887897331413. I check my bayescustomize.ini where there is : [Categorization] ham_cutoff = 0.10 spam_cutoff = 0.90 So, these parameters are not used by pop3proxy ? -- Recently using MacOSX.......

2 2

Re: egregious patents on anti-spam techniques (Kaitlin Duck Sherwood)
by Gary Robinson Jan. 31, 2003

Jan. 31, 2003

> Patent application on adaptive spam filtering: > <http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=/n… > ahtml/PTO/search-bool.html&r=3&f=G&l=50&co1=AND&d=PG01&s1=email.TTL.&OS=TTL/em > ail&RS=TTL/email> I looked at this last night. I am not a lawyer, so don't go to the bank on what I say. And I didn't spend a huge amount of time on it. But I do have some experience with patents, and I do understand the spambayes approach and the gist of their approach. It is my impression that the patent does not have a scope that encompasses Graham-derived filters, because they do not calculate "first" and "second" "symantic anchors" as the term is used in Claim 1. They seem to be trying to make a straightforward adaptation of technology that works well for classifying documents according to subject area, latent semantic analysis, into the spam realm. It would be very, very interesting to code and test their algorithm's performance against that of spambayes. One aspect of using latent semantic analysis is that it treats synonyms of known spammy words much as it does the spammy words themselves. It's sophisticated technology. But I'm not sure that its advantages matter much for spam detection with the kind of data we have available. It would be very interesting to know. --Gary -- [http://ThisURLEnablesEmailToGetThroughOverzealousSpamFilters.org] Gary Robinson CEO Transpose, LLC grobinson(a)transpose.com 207-942-3463 http://www.transpose.com http://radio.weblogs.com/0101454 > From: spambayes-request(a)python.org > Reply-To: spambayes(a)python.org > Date: Wed, 29 Jan 2003 17:50:54 -0500 > To: spambayes(a)python.org > Subject: Spambayes Digest, Vol 53, Issue 55 > > Send Spambayes mailing list submissions to > spambayes(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.python.org/mailman/listinfo/spambayes > or, via email, send a message with subject or body 'help' to > spambayes-request(a)python.org > > You can reach the person managing the list at > spambayes-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Spambayes digest..." > > > Today's Topics: > > 1. Re: Outlook plugin notes (Skip Montanaro) > 2. Re: Alpha 2 Release? (Richie Hindle) > 3. Re: Outlook plugin notes (Tim Stone - Four Stones Expressions) > 4. Re: Alpha 2 Release? (Francois Granger) > 5. Re: Alpha 2 Release? (Neale Pickett) > 6. Re: Alpha 2 Release? (Richie Hindle) > 7. Re: Alpha 2 Release? (Francois Granger) > 8. Re: Details in headers (G. Armour Van Horn) > 9. RE: Outlook plugin notes (Meyer, Tony) > 10. RE: Outlook plugin notes (Meyer, Tony) > 11. egregious patents on anti-spam techniques (Kaitlin Duck Sherwood) > > > ---------------------------------------------------------------------- > > Date: Wed, 29 Jan 2003 11:04:46 -0600 > From: Skip Montanaro <skip(a)pobox.com> > To: Neale Pickett <neale(a)woozle.org> > Cc: spambayes(a)python.org > Subject: Re: [Spambayes] Outlook plugin notes > Message-ID: <15928.2478.537965.516443(a)montanaro.dyndns.org> > In-Reply-To: <w53adhk2ahb.fsf(a)woozle.org> > References: <w53adhk2ahb.fsf(a)woozle.org> > Content-Type: text/plain; charset=us-ascii > MIME-Version: 1.0 > Content-Transfer-Encoding: 7bit > Precedence: list > Reply-To: skip(a)pobox.com > Message: 1 > > > Neale> One of our more tenacious tech writers ... > > You know when a company has good tech writers because their documentation is > head and shoulders above the competitions. I like to think of them as > librarians without the Donna Reed ("It's a Wonderful Life") personality. ;-) > > Good tech writers also make extremely good testers because they want the > documentation and the application to match exactly. > > Skip > > ------------------------------ > > Date: Tue, 28 Jan 2003 17:50:47 +0000 > From: Richie Hindle <richie(a)entrian.com> > To: spambayes(a)python.org > Subject: Re: [Spambayes] Alpha 2 Release? > Message-ID: <88gd3v4cj7v7qmdl28hofbj2ptih6hd13s(a)4ax.com> > In-Reply-To: <w53ptqi1ej4.fsf(a)woozle.org> > References: <a4233vsjdua1o8ufi2d1f8mei91o1h2eda(a)4ax.com> > <w53ptqi1ej4.fsf(a)woozle.org> > Content-Type: text/plain; charset=ISO-8859-1 > MIME-Version: 1.0 > Content-Transfer-Encoding: 8bit > Precedence: list > Reply-To: richie(a)entrian.com > Message: 2 > > > [Neale] >> Thanks a ton for putting a release together, Richie. > > No problem. I'm hoping to do this on Friday evening UK time, if that's OK > with everyone else? > > I'll fix the Mac OS X stack size problem for pop3proxy before the release - > I may not have time to do it "properly" by introducing a new > platform-dependent module, but we can munge things around afterwards. It's > more important to get a release out before the Linux Journal articles are > published, and it only seems to be the pop3proxy that has the problem. > > François, I haven't forgotten about you pop3proxy problem - if I have time > for a deeper investigation I'll do one. > > -- > Richie > > ------------------------------ > > Date: Wed, 29 Jan 2003 12:21:06 -0600 > From: Tim Stone - Four Stones Expressions <tim(a)fourstonesExpressions.com> > To: Neale Pickett <neale(a)woozle.org>, skip(a)pobox.com > Cc: spambayes(a)python.org > Subject: Re: [Spambayes] Outlook plugin notes > Message-ID: <WRGAFDN72MLPK7Q4398YUIHHE87HD.3e381b92@myst> > In-Reply-To: <15928.2478.537965.516443(a)montanaro.dyndns.org> > Content-Type: text/plain; charset="us-ascii" > MIME-Version: 1.0 > Precedence: list > Reply-To: tim(a)fourstonesExpressions.com > Message: 3 > > 1/29/2003 11:04:46 AM, Skip Montanaro <skip(a)pobox.com> wrote: > >> >> Neale> One of our more tenacious tech writers ... >> >> You know when a company has good tech writers because their documentation is >> head and shoulders above the competitions. I like to think of them as >> librarians without the Donna Reed ("It's a Wonderful Life") personality. ;-) >> >> Good tech writers also make extremely good testers because they want the >> documentation and the application to match exactly. > Think we could get her to write our doc? - TimS >> >> Skip >> >> _______________________________________________ >> Spambayes mailing list >> Spambayes(a)python.org >> http://mail.python.org/mailman/listinfo/spambayes >> >> > > > c'est moi - TimS > http://www.fourstonesExpressions.com > http://wecanstopspam.org > > > > ------------------------------ > > Date: Wed, 29 Jan 2003 20:02:18 +0100 > From: Francois Granger <francois.granger(a)free.fr> > To: richie(a)entrian.com > Cc: spambayes(a)python.org > Subject: Re: [Spambayes] Alpha 2 Release? > Message-ID: <a05200f2aba5dd54b1b98(a)[192.168.1.20]> > In-Reply-To: <88gd3v4cj7v7qmdl28hofbj2ptih6hd13s(a)4ax.com> > References: <a4233vsjdua1o8ufi2d1f8mei91o1h2eda(a)4ax.com> > <w53ptqi1ej4.fsf(a)woozle.org> <88gd3v4cj7v7qmdl28hofbj2ptih6hd13s(a)4ax.com> > Content-Type: text/plain; charset="iso-8859-1" ; format="flowed" > MIME-Version: 1.0 > Content-Transfer-Encoding: 8bit > Precedence: list > Message: 4 > > At 17:50 +0000 28/01/2003, in message Re: [Spambayes] Alpha 2 > Release?, Richie Hindle wrote: >> [Neale] >>> Thanks a ton for putting a release together, Richie. > > My thanks as well. > >> François, I haven't forgotten about you pop3proxy problem - if I have time >> for a deeper investigation I'll do one. > > No problem. > I did not saw the classification problem since the other day. It > seems that it is solved. > > I got a new fresh traceback tonight when I asked for review: > > Traceback (most recent call last): > > File "/Volumes/OS99/spambayes/spambayes/Dibbler.py", line 398, in > found_terminator > getattr(plugin, name)(**params) > > File "/Volumes/OS99/spambayes/pop3proxy.py", line 932, in onReview > self._appendMessages(page.table, messages, label) > > File "/Volumes/OS99/spambayes/pop3proxy.py", line 823, in _appendMessages > table += row > > File "/Volumes/OS99/spambayes/spambayes/PyMeldLite.py", line 787, in __iadd__ > nodes = self._nodeListFromSource(other) > > File "/Volumes/OS99/spambayes/spambayes/PyMeldLite.py", line 640, > in _nodeListFromSource > tree = _generateTree("<x>"+value+"</x>") > > File "/Volumes/OS99/spambayes/spambayes/PyMeldLite.py", line 574, > in _generateTree > g.feed(source) > > File "/Volumes/OS99/spambayes/spambayes/PyMeldLite.py", line 499, in feed > self._parser.Parse(data) > > File "/Volumes/OS99/spambayes/spambayes/PyMeldLite.py", line 529, > in StartElementHandler > newAttributes[str(name)] = self._unmungeEntities(str(value)) > > UnicodeEncodeError: 'ascii' codec can't encode character '\ue9' in > position 86: ordinal not in range(128) > > > > > -- > Recently using MacOSX....... > > ------------------------------ > > Date: Wed, 29 Jan 2003 12:38:21 -0800 > From: Neale Pickett <neale(a)woozle.org> > To: Francois Granger <francois.granger(a)free.fr> > Cc: spambayes(a)python.org > Subject: Re: [Spambayes] Alpha 2 Release? > Message-ID: <w53r8avzo76.fsf(a)woozle.org> > In-Reply-To: <a05200f2aba5dd54b1b98(a)[192.168.1.20]> (Francois Granger's > message of "Wed, 29 Jan 2003 20:02:18 +0100") > References: <a4233vsjdua1o8ufi2d1f8mei91o1h2eda(a)4ax.com> > <w53ptqi1ej4.fsf(a)woozle.org> > <88gd3v4cj7v7qmdl28hofbj2ptih6hd13s(a)4ax.com> > <a05200f2aba5dd54b1b98(a)[192.168.1.20]> > Content-Type: text/plain; charset=us-ascii > MIME-Version: 1.0 > Precedence: list > Message: 5 > > Francois Granger <francois.granger(a)free.fr> writes: > >> UnicodeEncodeError: 'ascii' codec can't encode character '\ue9' in >> position 86: ordinal not in range(128) > > Yeah, my wife's been getting those too. I'll look into her traceback. > > Yikes! I just swallowed the tine of a plastic fork! > > Neale > > ------------------------------ > > Date: Tue, 28 Jan 2003 21:41:33 +0000 > From: Richie Hindle <richie(a)entrian.com> > To: spambayes(a)python.org > Subject: Re: [Spambayes] Alpha 2 Release? > Message-ID: <c2ud3vk5hbtoukl1547q8eesvrfl3a7bbe(a)4ax.com> > In-Reply-To: <w53r8avzo76.fsf(a)woozle.org> > References: <a4233vsjdua1o8ufi2d1f8mei91o1h2eda(a)4ax.com> > <88gd3v4cj7v7qmdl28hofbj2ptih6hd13s(a)4ax.com> > <a05200f2aba5dd54b1b98(a)[192.168.1.20]> <w53r8avzo76.fsf(a)woozle.org> > Content-Type: text/plain; charset=ISO-8859-1 > MIME-Version: 1.0 > Content-Transfer-Encoding: 8bit > Precedence: list > Reply-To: richie(a)entrian.com > Message: 6 > > > [François] >> UnicodeEncodeError: 'ascii' codec can't encode character '\ue9' in >> position 86: ordinal not in range(128) > > This is bizarre. This is expat complaining that you can't have high-bit > characters in ASCII XML, which is quite right, but I replace all those > characters with charrefs on the way in: > >>>> def replaceHighCharacters(match): > ... return "&#%d;" % ord(match.group(1)) > ... >>>> re.sub('([\x80-\xff])', replaceHighCharacters, u"a b \xe9 c d") > u'a b é c d' > > So what's going on...? > > >> Yikes! I just swallowed the tine of a plastic fork! > > That'll teach you to try to get out of doing the washing up. 8-) > > -- > Richie Hindle > richie(a)entrian.com > > > ------------------------------ > > Date: Wed, 29 Jan 2003 22:40:03 +0100 > From: Francois Granger <francois.granger(a)free.fr> > To: spambayes(a)python.org > Subject: Re: [Spambayes] Alpha 2 Release? > Message-ID: <a05200f2bba5dfa5c91b6(a)[192.168.1.20]> > In-Reply-To: <w53r8avzo76.fsf(a)woozle.org> > References: <a4233vsjdua1o8ufi2d1f8mei91o1h2eda(a)4ax.com> > <88gd3v4cj7v7qmdl28hofbj2ptih6hd13s(a)4ax.com> > <a05200f2aba5dd54b1b98(a)[192.168.1.20]> <w53r8avzo76.fsf(a)woozle.org> > Content-Type: text/plain; charset="us-ascii" ; format="flowed" > MIME-Version: 1.0 > Precedence: list > Message: 7 > > At 12:38 -0800 29/01/2003, in message Re: [Spambayes] Alpha 2 > Release?, Neale Pickett wrote: >> Francois Granger <francois.granger(a)free.fr> writes: >> >>> UnicodeEncodeError: 'ascii' codec can't encode character '\ue9' in >>> position 86: ordinal not in range(128) >> >> Yeah, my wife's been getting those too. I'll look into her traceback. > > Your wife speaks some foreign language ? ;-) > >> Yikes! I just swallowed the tine of a plastic fork! > > My apologies for this, it is not _that_ important ;-) > > Thanks for all. > > -- > Recently using MacOSX....... > > ------------------------------ > > Date: Wed, 29 Jan 2003 14:11:55 -0800 > From: "G. Armour Van Horn" <vanhorn(a)whidbey.com> > Cc: spambayes(a)python.org > Subject: Re: [Spambayes] Details in headers > Message-ID: <3E3851AB.18018F9E(a)whidbey.com> > References: > <16E1010E4581B049ABC51D4975CEDB886199BD(a)UKDCX001.uk.int.atosorigin.com> > Content-Type: text/plain; charset=us-ascii > MIME-Version: 1.0 > Content-Transfer-Encoding: 7bit > Precedence: list > Reply-To: vanhorn(a)whidbey.com > Message: 8 > > Speaking as one who provides tech support for a hundred or so Windows > users, I find it perverse to put any file where Windows may change it or > obscure it. Against the flow from Redmond, I want my users to put their > data in folders they specifically control (normally on a file server, > never in "My Documents"). I want applications to put everything possible > in their respective directories, not in the registry, not in the current > equivalent of /windows/system. (And I always want to see all file > extensions!) > > I imagine that I'll end up putting some form of Spambayes on at least a > couple of dozen systems, so I'll get used to whatever is done, but I'd > strongly prefer that the file locations be easily understood and easily > learned so others don't have to spend so much time when a user messes an > installation up. > > Van > > "Moore, Paul" wrote: > >> From: Piers Haken [mailto:piersh@friskit.com] >>> I think that's a bit harsh. The directory is called >>> "Application Data", not "My Documents": it's designed >>> to be used by well-behaved applications only and it's >>> generally a bad idea for users to go mucking about with >>> stuff in there >> >> Sorry - I thought we were talking about the location of >> the INI file, which (at the moment, at least) is intended >> to be user editable. >> >> I've no problem with this location for purely application >> maintained configuration data. >> >> But I still think that there should at least be an option >> for the application directory to get deleted on uninstall - >> otherwise you get the same problem as with the registry of >> configuration data for uninstalled applications just getting >> left around and forgotten. >> >> Paul. >> >> _______________________________________________ >> Spambayes mailing list >> Spambayes(a)python.org >> http://mail.python.org/mailman/listinfo/spambayes > > -- > ---------------------------------------------------------- > Sign up now for Quotes of the Day, a handful of quotations > on a theme delivered every morning. > Enlightenment! Daily, for free! > mailto:twisted@whidbey.com?subject=Subscribe_QOTD > > For web hosting and maintenance, > visit Van's home page: http://www.domainvanhorn.com/van/ > ---------------------------------------------------------- > > > > ------------------------------ > > Date: Thu, 30 Jan 2003 11:32:01 +1300 > From: "Meyer, Tony" <T.A.Meyer(a)massey.ac.nz> > To: <spambayes(a)python.org> > Subject: RE: [Spambayes] Outlook plugin notes > Message-ID: <1ED4ECF91CDED24C8D012BCF2B034F1318CD37(a)its-xchg4.massey.ac.nz> > Content-Type: text/plain; > charset="iso-8859-1" > MIME-Version: 1.0 > Content-Transfer-Encoding: quoted-printable > Precedence: list > Message: 9 > >> * In her words, "When you filter to an online folder, SpamBayes >> automatically disables filtering when you connect offline. What I >> would like is that when I reconnect, SpamBayes should automatically >> reenable filtering and run it against those folders. Now I=20 >> have to do this manually." > I don't use any offline features, so I can't comment on this. > >> * Apparently Outlook comes with a "Junk Email" folder. > Hmm...does anyone else's Outlook have a "Junk Email" folder? Mine (2000 = > SR1) certainly didn't come with one. > >> * She feels end-users need more education about what "spam-possible" = > means. > That would be a documentation issue, right? And wasn't she a writer? = > .... > >> * The sliders in the configuration window should have tick marks. > I guess I would agree with that. I don't know who would even use the = > sliders when there's a text box just there, but ... > >> * In the anti-spam dialog box: >> o Enable filtering checkbox should be below filters, since you have >> to enable filtering before you can mess with the filters. > Agreed. > >> o The filters box needs a scrollbar, for those with a ton of = > folders >> to filter so you can see the text. > Or some other way of showing them all. A scrollbar would make it ugly, = > wouldn't it? > >> * Add a "spam column" in the anti-spam pulldown, so it's easy to add a >> new "spam %" column in the current folder view. > Is it possible to customise the current view via code? I wondered about = > doing this myself, since I seem to be constantly adding the column to = > new folders, but couldn't find any information about doing so (I must = > admit I didn't look that hard). > >> * She says that the plugin is definitely not filtering public folders. > This might be an issue that has been resolved (I'm not sure what version = > the release had). I'm checking this out on my system, but I don't have = > a lot of (mail) public folders (they're mostly calendars). I'll have to = > wait until one of them gets mail. > >> * She suggested deleting from public folders should go into a public >> spam folder. > Perhaps there could be an option to have mail from each folder you = > filter: > (a) go to the same uncertain/spam folders [as now] > (b) go to individual uncertain/spam folders [one set per filtered = > folder] > This would be quite a big interface change, though. Do people think = > it's worth it? > >> * All outbound mail should be trained as ham >> I really like this last one. I don't know if anyone's ever thought of >> training on outbound mail before. > Tim's post on this is in the November 2002 archive - "Bayes Training". = > The arguments against were: > * Because some spam is 'from' yourself, this deteriates the helpfulness = > of the from header. > * It's easy to find enough ham; much more deteriates the ratio. > > If a user saves their outgoing mail (in "Sent Items"), for example, then = > it's easy to train on that folder. I do this. > > =3DTony Meyer > > ------------------------------ > > Date: Thu, 30 Jan 2003 11:36:44 +1300 > From: "Meyer, Tony" <T.A.Meyer(a)massey.ac.nz> > To: <spambayes(a)python.org> > Subject: RE: [Spambayes] Outlook plugin notes > Message-ID: <1ED4ECF91CDED24C8D012BCF2B034F1318D3D7(a)its-xchg4.massey.ac.nz> > Content-Type: text/plain; > charset="iso-8859-1" > MIME-Version: 1.0 > Content-Transfer-Encoding: quoted-printable > Precedence: list > Message: 10 > >>> * Apparently Outlook comes with a "Junk Email" folder. >> Hmm...does anyone else's Outlook have a "Junk Email" folder? =20 >> Mine (2000 SR1) certainly didn't come with one. > > I take this back. If you use the Adult Mail/Junk Mail rules that = > Outlook offers, then if you chose the 'move mail' option and the 'junk = > mail' folder, then you are prompted to create a new "Junk Mail" folder. > > However, not every user will have one of these. I would suggest that = > the best option would be to change the documentation to suggest that _if = > there is one_ then to use the Junk Mail folder. I guess the plugin = > could check to see if there was an existing 'junk mail' folder and = > default to it, but then it could check for a 'spam' folder and default = > to that, too. Depends on what people are likely to have. > > =3DTony Meyer > > ------------------------------ > > Date: Wed, 29 Jan 2003 14:53:06 -0800 > From: Kaitlin Duck Sherwood <ducky(a)webfoot.com> > To: spambayes(a)python.org > Subject: [Spambayes] egregious patents on anti-spam techniques > Message-ID: <p0510030fba5e07ee9a74(a)[10.0.0.2]> > Content-Type: text/plain; charset=us-ascii; format=flowed > MIME-Version: 1.0 > Content-Transfer-Encoding: 7BIT > Precedence: list > Message: 11 > > Gang -- > > I've recently become aware of two egregious patent applications > related to spam fighting. The first one looks like it might > conceivably cover Bayesian filtering. It would be good if someone > more familiar with Bayesian/classifier/machine learning theory could > check it out and perhaps challenge ("protest") the application. > > The second is on using whitelists, blacklists, challenge-response, > and digital signatures to combat spam. I plan to protest that one > myself. I have killer prior art for whitelists, blacklists, and > challenge-response (see p.82 of _Stopping Spam_ by Schwartz & > Garfinkel, 1998). I do not know of prior art for using digital > signatures in the service of stopping spam. If you know of prior art > for that, you might want to issue a protest and/or send me the info. > > (If you send me prior art on digital signatures/spam, please > + read the patent claims first > + put PRIOR ART in the subject line.) > > I'm going to Japan for ten days, leaving Friday morning, and will not > have email connectivity then. > > To protest a patent, you need to file prior art (within 60 days!) > with the patent office. See: > http://www.uspto.gov/web/offices/pac/mpep/documents/1900.htm > and > http://www.uspto.gov/web/offices/pac/mpep/documents/0600_610.htm#sect610 > > Patent application on adaptive spam filtering: > <http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=/n… > ahtml/PTO/search-bool.html&r=3&f=G&l=50&co1=AND&d=PG01&s1=email.TTL.&OS=TTL/em > ail&RS=TTL/email> > > > Patent application on whitelists, blacklists, challenge-response, and > digital signatures used in spam-fighting: > <http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p > =1&u=/netahtml/PTO/srchnum.html&r=1&f=G&l=50&s1='20030009698'.PGNR.&OS=DN/20 > 030009698&RS=DN/20030009698> > > > > ------------------------------ > > _______________________________________________ > Spambayes mailing list > Spambayes(a)python.org > http://mail.python.org/mailman/listinfo/spambayes > > > End of Spambayes Digest, Vol 53, Issue 55 > ***************************************** >

6 7

Bayesian virus detection?
by Richard Jowsey Jan. 31, 2003

Jan. 31, 2003

I've "accidently" captured a handful of VB-script-type virii lately, since my spam training corpus apparently (luckily!) contained a few of these nasties. Got me thinking... I'd like to try training my classifier with a corpus of viral material, plus add a "virus" classification category into the mix, see what happens. However, I haven't a clue as to how to go about deliberately collecting such crud (and apparently, neither does Google)... Bloody weird question, but does anyone happen to have a collection of quarantined emails containing viral attachments, trojans, etc...? Which they could ZIP and send me for research purposes? Promise not to forward 'em to all my friends, ha ha... TIA, Richard PS: script kiddies and assorted black-hat hackers are officially invited to mail their nasty little efforts to richard(a)jowsey.com

6 7

Well done
by Francois Granger Jan. 31, 2003

Jan. 31, 2003

Spambayes is really astonishing. It flagged this mail with really few clues. Following, mail then clues ======================================== Return-Path: <dananunoaawf(a)aol.com> Delivered-To: pbarn(a)altern.org Received: (qmail 2141 invoked by alias); 31 Jan 2003 15:51:04 -0000 Received: from unknown (HELO aol.com) (213.42.188.253) by altern.org with SMTP; 31 Jan 2003 15:51:04 -0000 Message-ID: <000810a4ed04$baa75782$67542232(a)qdsbgnj.aye> From: <dananunoaawf(a)aol.com> To: <osa(a)altern.org> Cc: <osp(a)altern.org>, <oumpahpah(a)altern.org>, <ouplaboum(a)altern.org>, <pagano(a)altern.org>, <pagano2(a)altern.org>, <paimpol(a)altern.org>, <patate20(a)altern.org>, <patate22(a)altern.org>, <patoch(a)altern.org>, <patrickdombrowsky(a)altern.org>, <patrickv(a)altern.org>, <paul(a)altern.org>, <pbarn(a)altern.org> Subject: *lock in your 4 pct rate 123 2633BvYK4-226sJyQ3-17 Date: Fri, 31 Jan 2003 10:25:26 +0500 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_00C1_74D58D0A.C2060E16" X-Priority: 3 X-Mailer: Microsoft Outlook, Build 10.0.2616 Importance: Normal Status: X-Spambayes-Classification: spam <x-html><!x-stuff-for-pete base="" src="" id="0" charset="">% Pftjyealjgultwb <html><head><title>::M::</title></head><body><center><a href="http://211.157.100.107/mortgage/Lead236/index.htm"><img border="0" src="http://211.157.100.107/mortgage/p2X.bmp" width="427" height="252"> </a> </center><p><a target="_blank" href="re-move.htm"><font face="Arial" size="3">no more mail?</font></a></p> </body></html> X5P5lphu0ycTA7e7H3Na 2129PNyE3-553ZPfu8938Kml22</x-html> ======================================== Spam probability: 0.983622180161 *H* 0.00646291040099 *S* 0.973707270724 header:MIME-Version:1 0.298172028413 header:Return-Path:1 0.376864012565 header:Message-ID:1 0.385762811067 header:Importance:1 0.687752526214 content-type:multipart/mixed 0.714739667627 subject:123 0.844827586207 header:Received:2 0.852656659762 from:no real name:2**0 0.881348131394 subject:your 0.934782608696 x-mailer:microsoft outlook, build 10.0.2616 0.934782608696 from:addr:aol.com 0.95871559633 ======================================== -- Recently using MacOSX.......

1 0

Re: use as generic classifier? (not just spam/ham)
by Gary Robinson Jan. 31, 2003

Jan. 31, 2003

I've been thinking about that myself. I wonder if anyone else on this list has been? There are traditional classification techniques for classifying documents. Each document becomes a vector in n-space, and space is partitioned according to subject area. But that technology is typically complex and expensive. it would be interesting to take the whole spambayes spam/ham approach, and run it separately for every folder. Just do the same exact thing, but instead of the target being a junk mail folder, it would be a folder of some subject matter. So for each folder, instead of spam vs. nonspam, <that folder> vs. <not that folder> would be calculated. Then, the email would be allocated to the folder with the greatest certainty would be picked. That is, I wouldn't try to adapt the approach to work with more than a binary choice at a time. I don't think that would work (at least not with chi-square). But if you calculate the binary choice w/r/t each folder, the one with the most certainty can be chosen. It's a little bit ugly, if it gets the job done, so what. I think it would work. It might not work quite as well as in the spam/ham case, but it would be interesting to actually try it and see how well it did. It seems like the mods to spambayes to make it happen wouldn't be super-huge; if enough people felt that there might be some benefit, it seems like it might be worth doing... Just my 2 cents! > > I was wondering if spambayes could be used in a different way. Currently > I have a couple of users on a mailserver (imap+maildir). I'd like to > make it possible to have them create different subfolders where they > move mail to (from a mailinglist, about a specific subject, from a > specific person - whatever) through their standard imap client. Now, > when a new mail arrives on the mailserver, I'd like that mail to be > "scored" against all these folders and subfolders (maildirs), and for it > to be delivered to the folder that corresponds to the system's "best > guess". If the scores for all folders are within a certain (smallish) > range (percentage), the system's best guess is not good enough and it > should just be delivered to the standard inbox. --Gary -- [http://ThisURLEnablesEmailToGetThroughOverzealousSpamFilters.org] Gary Robinson CEO Transpose, LLC grobinson(a)transpose.com 207-942-3463 http://www.transpose.com http://radio.weblogs.com/0101454

1 0