Mailman 3 February 2004 - spambayes-dev

patch to improve statistics from spambayes
by Mark Moraes Feb. 29, 2004

Feb. 29, 2004

Hi. While I'm generally very happy with Spambayes, I was a bit confused by the statistics, which didn't seem to add up. I'm using Spambayes 1.0a9, the web page says SpamBayes POP3 Proxy Version 0.4 (February 2004) on Windows 2000 SP4. I'm using POP3 interface (tried a couple of different mail agents, including a command-line POP3 fetch, OE and Mozilla mail -- I see similar results, these are with the command line fetch). I have 'Lookup message in cache' set to yes, Notate to: unsure, Classify subject spam. I suppress caching of bulk ham. After a POP3 fetch, the Statistics page says: SpamBayes has processed 1150 messages - 754 (66%) good, 333 (29%) spam and 63 (5%) unsure. 324 messages were manually classified as good (0 were false positives). 379 messages were manually classified as spam (33 were false negatives). 6 unsure messages were manually identified as good, and 52 as spam. ** 1. 6 unsure good + 52 unsure spam adds up to 58. But the processed line says 63? It's not clear how many messages were manually reviewed/trained. ** 2. It's not clear that manually classified as good helps figure out what was accurately classified as good, because that includes ham, spam and unsures that were so classified. Ditto for spam. It's not clear how the 324 manually classified as good relate to the 754 good, and the 379 manually classified as spam relate to the 333 spam? And as a result, it's hard to estimate accuracy. ** 3. After using the Review web page to train and mark all 4 unsure as spam, 2 ham as spam and leaving all spam as-is (yay!), I see: SpamBayes has processed 1150 messages - 754 (66%) good, 333 (29%) spam and 63 (5%) unsure. 333 messages were manually classified as good (0 were false positives). 414 messages were manually classified as spam (35 were false negatives). 6 unsure messages were manually identified as good, and 56 as spam. The false positive count is clearly a bug, since I just classified 2 ham as spam, and I know I've done that often. But I've never had to classify spam as ham. Looks like fp & fn are inverted. The enclosed patch fixes that inversion, adds a few counters to tell which ham was manually identifed as spam and vice versa, as well as total ham/spam/manually reviewed, so one can calculate percentages. (The calculation is conservative; false positives/manually-reviewed ham, or false negatives/manually-reviewed spam, so that unreviewed messages don't skew the percentages) Also trimmed the statements somewhat to avoid over-long lines. (removed some verbs:-) Before the enclosed patch, Stats.py produces: SpamBayes has processed 1223 messages - 827 (68%) good, 333 (27%) spam and 63 (5%) unsure. 346 messages were manually classified as good (0 were false positives). 414 messages were manually classified as spam (35 were false negatives). 6 unsure messages were manually identified as good, and 56 as spam. With the patch, Stats.py produces: Classified 1223 messages - 827 (68%) ham, 333 (27%) spam and 63 (5%) unsure. Manually trained 760 messages: 340 of 375 ham messages manually confirmed (35 false positives 4.2%). 323 of 323 spam messages manually confirmed (0 false negatives 0.0%). Of 62 unsure messages, 6 (9.7%) manually identified as ham, 56 (90.3%) as spam. I find this much more useful -- hope you agree. Regards, Mark.

3 3

RE: [spambayes-dev] Problem with 1.0a9 Windows installer?
by Tony Meyer Feb. 27, 2004

Feb. 27, 2004

[Kenny] > OK, I've taken a look and it appears that registering > the addin using the outlook_addin_register executable > during install does two things differently than > registering the addin DLL directly with regsvr32: > * outlook_addin_register sets the DLL path in InprocServer32 > to "pythoncom23.dll", while regsvr32 sets it to > "<installation_path>\bin\outlook_addin.dll" Did you ever figure out what was causing this? I have an Excel plug-in that uses the same registration trickery as the Outlook plug-in's addin.py (guess where the code came from <wink>), and I ran into this problem today; i.e. pythoncom23.dll was the DLL path, rather than my excel_addin.dll. Weirdly, it was working before this, and I can't think of what code I changed that could effect this. In any case (and to bring it back to spambayes <wink>), I put a "pythoncom.frozen = True" line in after the "sys.frozendllhandle = ..." line (at the end of addin.py), and that fixed it. I presume that would fix it for you, too. I'm not sure why pythoncom.frozen wasn't already True, though, since it was running in a binary (py2exe 0.5.0, win32 200). Maybe Mark knows :) =Tony Meyer

4 8

FW: [Spambayes] Delete As Spam - Doesn't always work
by Tim Peters Feb. 27, 2004

Feb. 27, 2004

Does this log file ring bells with anyone? I haven't seen errors coming out of GetHTMLFromRTFProperty before. Geoff, you may have intended to attach the email message you couldn't delete as spam, but no such attachment arrived -- just the log file. [fwd'ed with permission] [Geoff Campbell] > Hi Tim - > > Attached is the latest log file (only one I could find), and also the > email message that I couldn't "Delete as Spam". In trying to solve > the problem, I upgraded to the latest SpamBayes version - that may > have wiped out the log of when the message originally came in. > > Anyway, thanks for you help. > > Geoff Campbell > > > -----Original Message----- > From: Tim Peters [mailto:tim.one@comcast.net] > Sent: Thursday, February 19, 2004 1:59 PM > To: geoff(a)controlg.com > Cc: spambayes(a)python.org > Subject: RE: [Spambayes] Delete As Spam - Doesn't always work > > > [Geoff Campbell] >> Sometimes (very rarely) when I select an obvious spam message and >> "click" on "Delete As Spam", nothing happens. It seems that the >> message is "SpamBayes proof"! This happens once every several days, >> but I can't see a pattern. Using Outlook 2002 (SP-1). Would be >> happy > >> to send you a representative email. > That would be good. There's probably a helpful (to us <wink>) message > in your SpamBayes log file whenever this happens. A real problem is > that Outlook destroys the exact structure of incoming email, so you > may not actually be able to give anyone else an email that reproduces > the problem. When "nothing happens" in an otherwise-working > SpamBayes, the usual cause is that the email is so badly formed > (violates so many standards about how email *should* be constructed) > that the SpamBayes email parser gives up trying to make any sense of > it. If that's what's happening, you will find helpful (to us) > information in your SpamBayes log file.

4 4

RE: [spambayes-dev] Re: Problem with 1.0a9 Windows installer?
by Tony Meyer Feb. 27, 2004

Feb. 27, 2004

> From what I have found in the code, setting pythoncom.frozen = True > seems to be safe. It would be interesting to know the values > of sys.frozen and pythoncom.frozen (if any) before. sys.frozen is 1, pythoncom.frozen is 0 (here, and for Kenny, because his print wouldn't have executed if bool(sys.frozen) wasn't True). > I wanted to make a bugfix release of py2exe - although this > fixes different bugs (I assume). If there's a problem with > COM registration I would like to wait until this is resolved. I think this is pretty specific to SpamBayes (and those like me that copy it <wink>). Surely there aren't many people around who are using this "pretend the exe is a dll" hack. Without that, all seems to be ok. OTOH, there have been three recent reports of this problem: """ Traceback (most recent call last): File "addin.pyc", line 1191, in OnConnection File "manager.pyc", line 908, in GetManager File "manager.pyc", line 344, in __init__ File "manager.pyc", line 492, in LocateDataDirectory File "win32com\shell\shell.pyc", line 9, in ? File "win32com\shell\shell.pyc", line 7, in __load ImportError: DLL load failed: A device attached to the system is not functioning. """ (Install goes fine, opening Outlook results in this). This looks like it is some sort of COM registration problem, but I don't know if it's with SpamBayes, win32com, or py2exe (I don't suppose your other bugfixes are for this? <wink>). One of the reporters has the same Windows version & Outlook version as me, and it works fine here, so it doesn't appear to be a result of that. I do have lots of python dlls scattered about the place, of course. =Tony Meyer

2 1

"Bayesian Dobly"
by Tony Meyer Feb. 27, 2004

Feb. 27, 2004

Has anyone else read the "Bayesian Dobly: Noise Reduction for Statistical Analysis" paper by Jonathan Zdziarski of nuclearelephant.com's DSPAM? (it was /.'d recently). The idea, basically, is to remove some tokens from the stream before classifying if they look like "noise" -i.e. junk words, or word salad. This is done by comparing the strength (distance from 0.5, for us) of words compared to their neighbours. Apparently it's quite successful for them. It doesn't appear that it would be all that difficult to implement a version of this for SpamBayes, so I thought I'd give it a go, unless someone here is going to tell me that it's not a good idea (or that they've already done it). (Obviously, if I do, I'll post the patch & results from testing here). =Tony Meyer

7 6

RE: [spambayes-dev] "Bayesian Dobly"
by Tony Meyer Feb. 27, 2004

Feb. 27, 2004

> On Thu, Feb 26, 2004 at 03:22:13PM -0600, Seth Goodman wrote: > > I think Dolby Labs would cringe at > > this, but Hormel got nabbed so why not Dolby? > > The trick is called "Dobly", not "Dolby". I don't think > "Dobly" is a trademarked word. Is that why it's named that? Like most other people here (it seems) I thought that it was misspelt, but it was consistent through the paper, so I used that (although when reading it in my head I 'said' "Dolby"). Maybe it's a new Harry Potter house elf <wink>. It doesn't seem to make much sense to use "Dobly" if "Dolby" is what they mean. I don't see (but IANAL) anything wrong with writing a paper that explains how "Dolby"-style techniques were used in email classification, assuming that it had (TM) and "Dolby is a registered trademark of ..." in all the right places. As long as they weren't going to use it to try and sell stuff. =Tony Meyer

3 3

RE: [spambayes-dev] Mac OS X package
by Tony Meyer Feb. 26, 2004

Feb. 26, 2004

> New to the list, and pretty new to spambayes. I've put together a > package for easy installation of spambayes on Mac OS X. > Posted it on the wiki; http://www.entrian.com/sbwiki/MacOSXPackage Cool, thanks! > However, the utilities (such as which_db etc) still look for the libs > in /System/Library/Python.framework/... and don't find them > (obviously). They should look for them wherever the config file says they are. A question is where the config file should default, to, though. On Windows, a bayescustomize.ini file is created in the "Application Data" directory, so everything defaults to being relative to that. There has been some discussion (although it's not the case yet) of making Linux default to creating a .spambayesrc file in ~, or maybe ~/.spambayes. What's the "correct" place with OS X? (Note that this includes the bayescustomize.ini file (prefs), as well as the two databases, and the cache directories (user data), so it could be quite big). Does anyone know if there's a way to get the path to this, like the win32all function that provides the "Application Data" directory? > the utilities, but I'm still wondering why everything else checks the > current directory first and then elsewhere for it's modules etc. when > the testtools and utilities don't...? The way it works is that all the scripts look for a bayescustomize.ini file - they look in an environment variable (BAYESCUSTOMIZE), in ~ (for .spambayesrc) and in the current working directory. If that's found, then all default file locations are relative to that. If it's not found then (unless win32 as above) the current working directory is used as a default location, and everything is relative to that. I suspect that in your testing, you had a bayescustomize.ini file in your scripts directory, and not elsewhere, which would explain the differing behaviour. > And also, am I breaching the license by leaving them out? The license pretty much lets you do whatever you want with the code, although IANAL. =Tony Meyer --- Please always include the list (spambayes(a)python.org) in your replies (reply-all), and please don't send me personal mail about SpamBayes. This way, you get everyone's help, and avoid a lack of replies when I'm busy.

4 3

Re: Mac OS X package
by Thomas Juntunen Feb. 26, 2004

Feb. 26, 2004

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 02/22/04, Sam Thorne <sam(a)s-j-t.co.uk> imposed order on a stream of electrons to say: >New to the list, and pretty new to spambayes. I've put together a >package for easy installation of spambayes on Mac OS X. Just for completeness, I thought I'd mention the MacPython PackageManager contains a spambayes package (version 1.0a7, IIRC) that will install automatically and try to satisfy any dependencies. The maintainers may be interested in what you're doing. MacPython: http://homepages.cwi.nl/~jack/macpython/ To get the spambayes package, you need to use Bob Ippolito's extended package database: http://undefined.org/python/pimp/ HTH, Thomas Juntunen -----BEGIN PGP SIGNATURE----- Version: PGP SDK 3.0 iQA/AwUBQD1BZtFoei/9T3YdEQJSdQCePfQZgcwQ89KKHx+Y0XJffEiBrtoAnR1Y dQHK2mEmJNDPdNt7gpIRfJfB =pEhI -----END PGP SIGNATURE-----

1 0

RE: [spambayes-dev] getting rid of the "new mail" icon
by antoine.trux＠nokia.com Feb. 24, 2004

Feb. 24, 2004

Tony, > You could look into InBoxer (there's a link on our 'related' > page). It's > the SpamBayes code (or close enough), so you'll get the same > results. I > believe they have done *something* to address this, although > I'm not sure > exactly what. In any case, since you'd be giving them money, > you'd be able > to ask them for features, if they don't do what you want already. Thank you very much for pointing me at InBoxer. I actually placed my order yesterday. Yes, they did something to address this: they have their own icon (this is actually the solution that Glenn Brown suggested in a message that he posted last Thursday). Antoine

1 0

Urgent, please help
by Stephen Luk Feb. 24, 2004

Feb. 24, 2004

I have to say Spambayes is a great product, and I've been using it happily. There's one problem that I have found out, the Spambayes filters my inbox emails before my Outlook(2003) client rules are applied, is there a way to check new emails with my own outlook rules before the SpamBayes filter applies? Thanks for your help steve

1 0