Mailman 3 December 2003 - spambayes-dev

A URL experiment
by Tim Peters Jan. 6, 2004

Jan. 6, 2004

Over on the spambayes list yesterday, we were discussing a particularly good identity-theft scam spam, purporting to be from PayPal. It linked extensively to PayPal's real site, and about the only fishy lexical thing was a highly obfuscated href (full of % escapes). We don't do anything special with % escapes in URLs now. Maybe we should. The attached patch does. I don't have enough personal email saved to make for a good test, but who cares <wink>. I just took what I had, slammed into randomly into 10 even sets, and did "the usual" cross-validation business on it. All of this email is less than a week old, is all the email I've gotten since then, is atypical for me (Christmas time -> a lot less email than usual, but a spike in personal email), and runs 3:1 in favor of ham. None of that matters, though -- *whatever* you have, and however you train, the interesting question is just how it does with the patch, compared to without it. I ran my 10-fold CV with "the default" settings for Outlook. These match the current (CVS) project defaults, with the addition of [Tokenizer] replace_nonascii_chars: True record_header_absence: True I'm *not* using mine_received_headers or x-use_bigrams in these tests. befores -> afters -> <stat> tested 151 hams & 52 spams against 1359 hams & 468 spams [19 repetitions of that] false positive percentages 0.000 0.000 tied 0.000 0.000 tied 0.000 0.000 tied 0.000 0.000 tied 0.000 0.000 tied 0.000 0.000 tied 0.662 0.662 tied 0.000 0.000 tied 0.000 0.000 tied 0.000 0.000 tied won 0 times tied 10 times lost 0 times total unique fp went from 1 to 1 tied mean fp % went from 0.0662251655629 to 0.0662251655629 tied false negative percentages 1.923 1.923 tied 0.000 0.000 tied 0.000 0.000 tied 0.000 0.000 tied 0.000 0.000 tied 0.000 0.000 tied 1.923 1.923 tied 1.923 1.923 tied 0.000 0.000 tied 0.000 0.000 tied won 0 times tied 10 times lost 0 times total unique fn went from 3 to 3 tied mean fn % went from 0.576923076924 to 0.576923076924 tied ham mean ham sdev 0.44 0.44 +0.00% 4.52 4.52 +0.00% 0.34 0.34 +0.00% 4.11 4.11 +0.00% 0.27 0.27 +0.00% 3.16 3.16 +0.00% 0.17 0.17 +0.00% 1.51 1.51 +0.00% 1.06 1.06 +0.00% 9.11 9.12 +0.11% 0.00 0.00 +(was 0) 0.01 0.01 +0.00% 0.78 0.78 +0.00% 8.16 8.16 +0.00% 0.42 0.43 +2.38% 5.19 5.21 +0.39% 0.01 0.01 +0.00% 0.11 0.11 +0.00% 0.07 0.07 +0.00% 0.90 0.90 +0.00% ham mean and sdev for all runs 0.36 0.36 +0.00% 4.77 4.78 +0.21% spam mean spam sdev 96.41 96.43 +0.02% 13.52 13.51 -0.07% 98.51 98.56 +0.05% 6.99 6.99 +0.00% 97.80 97.80 +0.00% 6.42 6.41 -0.16% 98.21 98.22 +0.01% 7.31 7.30 -0.14% 93.00 93.03 +0.03% 16.68 16.66 -0.12% 97.40 97.41 +0.01% 8.29 8.27 -0.24% 97.58 97.70 +0.12% 12.30 12.18 -0.98% 97.01 97.02 +0.01% 14.38 14.37 -0.07% 95.90 96.03 +0.14% 11.61 11.46 -1.29% 98.86 98.86 +0.00% 6.12 6.11 -0.16% spam mean and sdev for all runs 97.07 97.11 +0.04% 11.09 11.05 -0.36% ham/spam mean difference: 96.71 96.75 +0.04 Not much to talk about there! Pretty much indistinguishable, although the spam mean went up a tad consistently, and the spam sdev down a tad consistently. table.py's "best cost" output shows that I could have reduced the optimal cost by 1 unsure if I changed my cutoffs: filename: before after ham:spam: 1510:520 1510:520 fp total: 1 1 fp %: 0.07 0.07 fn total: 3 3 fn %: 0.58 0.58 unsure t: 39 39 unsure %: 1.92 1.92 real cost: $20.80 $20.80 best cost: $17.60 $17.40 h mean: 0.36 0.36 h sdev: 4.77 4.78 s mean: 97.07 97.11 s sdev: 11.09 11.05 mean diff: 96.71 96.75 k: 6.10 6.11 So the change would have been the tiniest of wins for me. For you? BTW, the fp here was an "end of year sale" blaring HTML ad from Gateway. That's ham to me, but there are no other msgs from Gateway in this email. It contains enough Gateway-specific lexicalisms that training on one is enough to score future ones as solid ham. The PayPal scam that started this remained a solid FN.

3 12

Re: [Spambayes-checkins] spambayes/scripts sb_server.py, 1.15, 1.16
by Richie Hindle Jan. 2, 2004

Jan. 2, 2004

[Tony] > Modified Files: > sb_server.py > Log Message: > When we stopped sb_server and then restarted, we didn't init the state, so it > wouldn't work. Fix that. > > [...] > > def prepare(): > + state.init() > state.prepare() This edit keeps appearing and disappearing. Mark removed that line in order to fix the fact that none of sb_server's command line arguments worked. Tony has now put it back in order to fix a restart problem, which has once again broken all the command line arguments. The docstring for State.__init__() describes how the code was originally intended to work: """Initialises the State object that holds the state of the app. The default settings are read from Options.py and bayescustomize.ini and are then overridden by the command-line processing code in the __main__ code below.""" The __main__ code is now in a function called run(), and the code to read the options is now in State.init(). Calling State.init() a second time, as prepare() now does, overwrites the command line options set up by run(). It does seem weird that there's a State.prepare() and a global prepare(), but the global prepare() calls both State.init() and State.prepare(). I don't know much about this code (if it was checked in under my name, it must have been my evil twin that wrote it 8-) Does anyone have a clear idea of what each of prepare(), State.__init__(), State.init(), and State.prepare() are all intended to do? I think the command line option code needs to be inserted somewhere into one of them, but I'm not 100% sure what each of them is for. PS. (next eight hours): Mark: You are too drunk to reply. PS. (after eight hours): Mark: How's the head? 8-) -- Richie Hindle richie(a)entrian.com

2 1

RE: [spambayes-dev] Experimental SpamBayes build available
by Tony Meyer Jan. 2, 2004

Jan. 2, 2004

[Skip] > Thanks for the new installer. I tried it out on my > little-used Win2k machine. While it seemed to install fine, > the tray icon does nothing but briefly change the pointer to > an hourglass. I double-clicked the sb_server.exe icon and it > popped up a window then immediately went away. In your temp directory (C:\Documents and Settings\[username]\Local Settings\Temp in Win2k, I think) there should be some SpamBayesServerN.log files (where N is a number). Could you grab any that are there and mail them to me/the list? (I haven't seen this myself, so it would be good to figure out what it is). =Tony Meyer

2 1

RE: [spambayes-dev] Experimental SpamBayes build available
by Tony Meyer Jan. 1, 2004

Jan. 1, 2004

> I really don't like the code in Options.py that handles the > default values for these storage items. I'm not sure it is > to blame, but it did cause me to see a new .db file created > in the cwd, rather than the data directory - as my INI file > already existed, it didn't get the default FQN for the new option. The idea was that if you already had an ini file, then you had already set things up (you were an existing user), and so we wouldn't want to fiddle with your setup, whatever it was, because that might mean that we lose track of your databases. (This could even be with the default "hammie.db" in the cwd for persistent_storage_file, if the user always runs the script from the same directory). Part of the problem is that when consolidating the storage name options, I picked "hammie.db" over "~/.hammiedb", which was by far the better option. I didn't realise that it would expand quite nicely on Windows (well, 2k and XP; I presume earlier as well), and I presume on Macs as well. > IMO, the ini files should generally store relative path > names, being relative to the directory of the config file > being used. +1. I can go through and check this in if you like (once others have ripped into the idea ;). > Code speaks louder than words - I'm suggesting: [...] > The default remains "statistics_database.db". The proper default, of course, is still "hammie.db". When I put that code in to put things into a better place for Windows users I figured that since they wouldn't have an existing db, a more easily understandable name would be good, too, but I didn't think that I could change it for everyone. Do we continue setting this option to "statistics_database.db" in that place? (Without the FQN). The same code also gives new default names for the cache directories and messageinfo db. > All code that uses this option > ('persistent_storage_file') does so via a new function: Presumably also these: [URLRetriever] x-cache_directory [Storage] messageinfo_storage_file [Storage] spam_cache [Storage] ham_cache [Storage] unknown_cache What about these? [TestDriver] spam_directories [TestDriver] ham_directories > def get_pathname_option(section, value): > filename = options.get(section, value) > if not os.path.isabs(filename): > return filename Shouldn't that be "if os.path.isabs(filename):"? > # maybe expanduser() to *nix? I think this is necessary, yes, but before this (because os.path.isabs("~/hammie.db") is False). What about this? def get_pathname_option(section, value): filename = os.path.expanduser(options.get(section, value)) if os.path.isabs(filename): return filename return os.path.join(os.path.dirname(optionsPathname), # existing global filename) How do people feel about having this happen implictly when one of these options is used, rather than explicitly? (I worry that we'll miss an occurrence of one of them, or that someone (maybe me!) will add new code and forget to use the get_pathname_option function). Something like this: [Current code in OptionsClass.py] def get(self): '''Get option value.''' return self.value [Proposed] def expand_path(value): filename = os.path.expanduser(value) if os.path.isabs(filename): return filename return os.path.join(os.path.dirname(optionsPathname), # existing global filename) def get(self): '''Get option value. If the option is a path, then get relative to the configuration file.''' if self.allowed_values in [PATH, FILE_WITH_PATH,]: # maybe also VARIABLE_PATH? return self.expand_path(self.value) return self.value =Tony Meyer

2 1

RE: [spambayes-dev] Re: [Spambayes-checkins] spambayes/scriptssb_server.py, 1.15, 1.16
by Tony Meyer Dec. 31, 2003

Dec. 31, 2003

> This edit keeps appearing and disappearing. Mark removed > that line in order to fix the fact that none of sb_server's > command line arguments worked. Tony has now put it back in > order to fix a restart problem, which has once again broken > all the command line arguments. Opps. Sorry, I must try and pay more attention... > Does anyone have a clear idea of what each of prepare(), > State.__init__(), State.init(), and State.prepare() are all > intended to do? I think the command line option code needs > to be inserted somewhere into one of them, but I'm not 100% > sure what each of them is for. I'm not sure about intent, but this is what they currently do: __init__(): calls init() init(): opens the log file sets up the list of servers/ports loads options from configuration file resets statistics prepare(): opens mutex (and in calling createWorkers()) opens db opens the corpora creates the trainers Should init() be only done once for every time that sb_server is run, and prepare() each time it is started/stopped? In that case it should be: __init__(): calls init() init(): opens the log file loads options from configuration file resets statistics prepare(): opens mutex sets up the list of servers/ports (and in calling createWorkers()) opens db opens the corpora creates the trainers This makes prepare() a kind of anti-close(). This is probably the fix I should have applied - moving setting up the list of servers/ports to prepare() instead of calling init() again. It needs to be done *sometime* after close(), though, and before the call to createWorkers(). Whether the log file and statistics should be reset on start/stop, I don't know, but I suspect not. This would mean that the command-line code could be anywhere between __init__() and prepare(). +1 to docstrings for init() and prepare(), though :) =Tony Meyer

1 0

Re: [spambayes-dev] A URL experiment
by T. Alexander Popiel Dec. 31, 2003

Dec. 31, 2003

In message: <16370.22611.419923.477159(a)montanaro.dyndns.org> Skip Montanaro <skip(a)pobox.com> writes: > >There's the rub. What might be really good ideas at this point will >probably only result in very small changes in performance because the >baseline system is currently so good. Aye, I think that's what killed this sort of tokenizer/classifier testing a year ago... - Alex

1 0

Re: [spambayes-dev] A URL experiment
by T. Alexander Popiel Dec. 31, 2003

Dec. 31, 2003

In message: <16370.22753.639278.165195(a)montanaro.dyndns.org> Skip Montanaro <skip(a)pobox.com> writes: > > Tony> 3-way compare: > > Tony> filename: bases nntims pickskips > Tony> ham:spam: 3668:4099 3668:4099 > Tony> 3668:4099 > Tony> fp total: 6 6 6 > Tony> fp %: 0.16 0.16 0.16 > Tony> fn total: 20 20 20 > ... > >What do you use to generate these three-way comparisons? That's table.py output. - Alex

1 0

Semi-results for TOE, TOAE, and expiry
by T. Alexander Popiel Dec. 31, 2003

Dec. 31, 2003

Yes, a few days ago I promised a further report on various things, including the effects of alternate start points in my dataset and expiry on the train_on_everything and train_on_almost_everything regimes. Unfortunately, all I have at the moment is some preliminary results and a big heap of frustration pointed at my computer. 1. There doesn't appear to be anything particularly magical about 120 days after start. Rotating my data forward or backward 80 days shows that (a) there was a particular event/change in my data at about 120 days after I started collecting that affects the accuracy of further classifications, and (b) the general curve of getting better for a few months then decaying for the rest of time still holds even when the data is rotated... but the curve is not as distinct when not reinforced by (a). 2. Expiry (as I implemented it) appears to be a very bad thing for long-term TOAE. I implemented it to expire trained messages after 120 days, without completely rebuilding the classifier. This resulted in significantly degraded accuracy after about 250 days, though that may just be due to an ever increasing spam/ham imbalance. There was a sharp drop in the amount of spam training for about 30 days after the initial expiry date, and then a net spam training rate about equivalent to non-expiring TOAE until the "latest windows update" worm, after which spam training about doubled the non-expiry version. This seems to show that spam mutation has a stong effect on 4-month expiry for TOAE. On the other hand, net ham training was fairly consistently slightly negative after expiry commenced, showing that once it got a good idea of what ham was and threw out the oddballs that got trained on initially, it didn't need much categorize ham. By the end of the mess (at 418 days) the spam:ham ratio was over 15:1, and the unsure rate was around 3% (compared to non-expiring with 4.5:1 and 1%). 3. Expiry for TOE seems neutral (compared to non-expiring TOE), to the best of my ability to eyeball the three runs that actually completed. The graphs I have are at: http://www.wolfskeep.com/~popiel/spambayes/plots/expire.html My primary machine (cashew.wolfskeep.com) unfortunately doesn't seem have the capability to maintain reliable service while running these tests anymore. They're just too big, and it doesn't have the memory/CPU to do everything all at once (including running my web server, a mysql engine, my mail feed, etc.). Plus, it appears that Linux 2.4.18 doesn't take too kindly to multiple processes trying to access/manipulate a single directory with over 100,000 files in it; anything that touches that directory after things have started going wonky just hangs in disk-wait. I'm suspecting a deadlock in the filesystem layer on extended directory operations... probably due to not enough file cache (see my memory problems) to hold the entire structure at once. I haven't poked deep enough into the ext2 drivers to be sure, though. Anyway, I'm not going to be able to do all that much until I get this straightened out. I'll add graphs and stuff to the wiki as I have time, but that's likely to be all for a bit... - Alex

1 0

RE: [spambayes-dev] Experimental SpamBayes build available
by Tony Meyer Dec. 31, 2003

Dec. 31, 2003

[I'll leave the install stuff for Mark, but I can sort out the rest of these]. > The ini file for the proxy appeared in "C:\Documents and > Settings\rjh\Application Data\SpamBayes\Proxy" as you'd > expect, but the database and cache directories appeared in > "C:\Program Files\SpamBayes\bin". Did the ini file have the appropriate [Storage] lines in it? It's meant to add them in there, storing the directories in that directory, too. You didn't already have an ini file in there, did you? (It only adds those lines if it's a new file, so that it doesn't overwrite someone's settings). > I'd question whether > we need the Stop/Start command - why would I want the tray > icon to stay there but the application to not run? I was thinking this just yesterday. I'm not sure what the original reasoning behind having it was (and it may have been me that put it there ;). +1 to getting rid of it, unless someone does know the reasoning. We can dump the 'stopped' icon, then, too. (I'd like to see a '!' icon, though, which appeared when there were important status messages to review). > After training through the web interface, the home page still > says "Database has no training information ..." even though > the stats say "Total emails trained: Spam: 3 Ham: 18". Good spotting. I've checked in a fix for this. > Defaulting the "Maximum results" field in the Find pane to 1 > seems wrong. It made sense when all you could do was search > for a message ID (because they're unique) but if I'm > searching for text, I'll want to see all the hits. Fair enough. Line 435 of ui.html; change it to whatever you like most :) > The Find pane only looks in the unknown cache, so it won't > find anything once you've trained. It ought to look in the > ham and spam caches as well. Are you positive? The code has it looking in all three, and a quick test here had it finding messages in more than one. > I deliberately induced a false positive (by training on a > thousand spams with no hams trained) then corrected it via > the Review page, and the statistics now say "1 being false > negatives" (plural: ack!) and "0 being false positives". > That's the wrong way round. Opps, my bad. I've checking in a fix for this. I think I've fixed all the plurals, too. If you've still got that false positive statistic around, could you give it a run from cvs? =Tony Meyer

3 3

RE: [spambayes-dev] A URL experiment
by Tony Meyer Dec. 31, 2003

Dec. 31, 2003

My results (this is with a chuck of my most recent mail, with timcv.py -n10). Tim's patch: bases.txt -> nntims.txt -> <stat> tested 357 hams & 395 spams against 3311 hams & 3704 spams -> <stat> tested 397 hams & 384 spams against 3271 hams & 3715 spams -> <stat> tested 385 hams & 433 spams against 3283 hams & 3666 spams -> <stat> tested 407 hams & 397 spams against 3261 hams & 3702 spams -> <stat> tested 350 hams & 412 spams against 3318 hams & 3687 spams -> <stat> tested 338 hams & 405 spams against 3330 hams & 3694 spams -> <stat> tested 359 hams & 416 spams against 3309 hams & 3683 spams -> <stat> tested 358 hams & 405 spams against 3310 hams & 3694 spams -> <stat> tested 348 hams & 411 spams against 3320 hams & 3688 spams -> <stat> tested 369 hams & 441 spams against 3299 hams & 3658 spams -> <stat> tested 357 hams & 395 spams against 3311 hams & 3704 spams -> <stat> tested 397 hams & 384 spams against 3271 hams & 3715 spams -> <stat> tested 385 hams & 433 spams against 3283 hams & 3666 spams -> <stat> tested 407 hams & 397 spams against 3261 hams & 3702 spams -> <stat> tested 350 hams & 412 spams against 3318 hams & 3687 spams -> <stat> tested 338 hams & 405 spams against 3330 hams & 3694 spams -> <stat> tested 359 hams & 416 spams against 3309 hams & 3683 spams -> <stat> tested 358 hams & 405 spams against 3310 hams & 3694 spams -> <stat> tested 348 hams & 411 spams against 3320 hams & 3688 spams -> <stat> tested 369 hams & 441 spams against 3299 hams & 3658 spams false positive percentages 0.000 0.000 tied 0.000 0.000 tied 0.000 0.000 tied 0.246 0.246 tied 0.000 0.000 tied 0.000 0.000 tied 0.557 0.557 tied 0.559 0.559 tied 0.287 0.287 tied 0.000 0.000 tied won 0 times tied 10 times lost 0 times total unique fp went from 6 to 6 tied mean fp % went from 0.164881884948 to 0.164881884948 tied false negative percentages 0.253 0.253 tied 0.781 0.781 tied 0.462 0.462 tied 0.756 0.756 tied 0.243 0.243 tied 0.247 0.247 tied 0.240 0.240 tied 0.494 0.494 tied 0.973 0.973 tied 0.454 0.454 tied won 0 times tied 10 times lost 0 times total unique fn went from 20 to 20 tied mean fn % went from 0.490257037938 to 0.490257037938 tied ham mean ham sdev 1.18 1.17 -0.85% 7.76 7.67 -1.16% 0.99 0.99 +0.00% 6.64 6.64 +0.00% 0.84 0.85 +1.19% 6.14 6.14 +0.00% 1.99 2.10 +5.53% 9.46 9.73 +2.85% 0.49 0.49 +0.00% 3.59 3.57 -0.56% 0.85 0.87 +2.35% 5.45 5.46 +0.18% 1.16 1.16 +0.00% 9.30 9.29 -0.11% 1.20 1.30 +8.33% 8.13 8.66 +6.52% 1.55 1.55 +0.00% 8.05 8.05 +0.00% 0.47 0.47 +0.00% 3.22 3.15 -2.17% ham mean and sdev for all runs 1.08 1.10 +1.85% 7.13 7.21 +1.12% spam mean spam sdev 98.75 98.75 +0.00% 8.72 8.72 +0.00% 97.67 97.69 +0.02% 11.26 11.24 -0.18% 98.08 98.14 +0.06% 10.12 9.97 -1.48% 98.16 98.16 +0.00% 10.19 10.20 +0.10% 98.35 98.41 +0.06% 8.77 8.69 -0.91% 98.45 98.47 +0.02% 8.97 8.86 -1.23% 98.35 98.41 +0.06% 9.73 9.69 -0.41% 98.25 98.36 +0.11% 9.16 8.96 -2.18% 97.93 97.97 +0.04% 11.99 11.98 -0.08% 98.92 98.93 +0.01% 7.62 7.62 +0.00% spam mean and sdev for all runs 98.30 98.34 +0.04% 9.72 9.66 -0.62% ham/spam mean difference: 97.22 97.24 +0.02 Skip's patch: bases.txt -> pickskips.txt -> <stat> tested 357 hams & 395 spams against 3311 hams & 3704 spams -> <stat> tested 397 hams & 384 spams against 3271 hams & 3715 spams -> <stat> tested 385 hams & 433 spams against 3283 hams & 3666 spams -> <stat> tested 407 hams & 397 spams against 3261 hams & 3702 spams -> <stat> tested 350 hams & 412 spams against 3318 hams & 3687 spams -> <stat> tested 338 hams & 405 spams against 3330 hams & 3694 spams -> <stat> tested 359 hams & 416 spams against 3309 hams & 3683 spams -> <stat> tested 358 hams & 405 spams against 3310 hams & 3694 spams -> <stat> tested 348 hams & 411 spams against 3320 hams & 3688 spams -> <stat> tested 369 hams & 441 spams against 3299 hams & 3658 spams -> <stat> tested 357 hams & 395 spams against 3311 hams & 3704 spams -> <stat> tested 397 hams & 384 spams against 3271 hams & 3715 spams -> <stat> tested 385 hams & 433 spams against 3283 hams & 3666 spams -> <stat> tested 407 hams & 397 spams against 3261 hams & 3702 spams -> <stat> tested 350 hams & 412 spams against 3318 hams & 3687 spams -> <stat> tested 338 hams & 405 spams against 3330 hams & 3694 spams -> <stat> tested 359 hams & 416 spams against 3309 hams & 3683 spams -> <stat> tested 358 hams & 405 spams against 3310 hams & 3694 spams -> <stat> tested 348 hams & 411 spams against 3320 hams & 3688 spams -> <stat> tested 369 hams & 441 spams against 3299 hams & 3658 spams false positive percentages 0.000 0.000 tied 0.000 0.000 tied 0.000 0.000 tied 0.246 0.246 tied 0.000 0.000 tied 0.000 0.000 tied 0.557 0.557 tied 0.559 0.559 tied 0.287 0.287 tied 0.000 0.000 tied won 0 times tied 10 times lost 0 times total unique fp went from 6 to 6 tied mean fp % went from 0.164881884948 to 0.164881884948 tied false negative percentages 0.253 0.253 tied 0.781 0.781 tied 0.462 0.462 tied 0.756 0.756 tied 0.243 0.243 tied 0.247 0.247 tied 0.240 0.240 tied 0.494 0.494 tied 0.973 0.973 tied 0.454 0.454 tied won 0 times tied 10 times lost 0 times total unique fn went from 20 to 20 tied mean fn % went from 0.490257037938 to 0.490257037938 tied ham mean ham sdev 1.18 1.18 +0.00% 7.76 7.76 +0.00% 0.99 0.99 +0.00% 6.64 6.64 +0.00% 0.84 0.84 +0.00% 6.14 6.14 +0.00% 1.99 1.99 +0.00% 9.46 9.46 +0.00% 0.49 0.50 +2.04% 3.59 3.60 +0.28% 0.85 0.87 +2.35% 5.45 5.55 +1.83% 1.16 1.16 +0.00% 9.30 9.30 +0.00% 1.20 1.21 +0.83% 8.13 8.14 +0.12% 1.55 1.55 +0.00% 8.05 8.06 +0.12% 0.47 0.47 +0.00% 3.22 3.22 +0.00% ham mean and sdev for all runs 1.08 1.08 +0.00% 7.13 7.14 +0.14% spam mean spam sdev 98.75 98.78 +0.03% 8.72 8.56 -1.83% 97.67 97.70 +0.03% 11.26 11.25 -0.09% 98.08 98.08 +0.00% 10.12 10.12 +0.00% 98.16 98.17 +0.01% 10.19 10.15 -0.39% 98.35 98.38 +0.03% 8.77 8.73 -0.46% 98.45 98.46 +0.01% 8.97 8.97 +0.00% 98.35 98.38 +0.03% 9.73 9.68 -0.51% 98.25 98.29 +0.04% 9.16 9.05 -1.20% 97.93 97.95 +0.02% 11.99 11.98 -0.08% 98.92 98.93 +0.01% 7.62 7.62 +0.00% spam mean and sdev for all runs 98.30 98.32 +0.02% 9.72 9.68 -0.41% ham/spam mean difference: 97.22 97.24 +0.02 3-way compare: filename: bases nntims pickskips ham:spam: 3668:4099 3668:4099 3668:4099 fp total: 6 6 6 fp %: 0.16 0.16 0.16 fn total: 20 20 20 fn %: 0.49 0.49 0.49 unsure t: 178 173 175 unsure %: 2.29 2.23 2.25 real cost: $115.60 $114.60 $115.00 best cost: $93.00 $91.20 $92.40 h mean: 1.08 1.10 1.08 h sdev: 7.13 7.21 7.14 s mean: 98.30 98.34 98.32 s sdev: 9.72 9.66 9.68 mean diff: 97.22 97.24 97.24 k: 5.77 5.76 5.78 Rather like Tim's results, really, at least to my ignorant eyes. =Tony Meyer

3 2