[Spambayes] Need testers!

[Neale Pickett]

So then, Tim Peters <tim.one@comcast.net> is all like:

> it would help if you did this all over again.  It was a pure and
> significant win for both error rates in this test report, but that may
> change after msgs are shuffled between the spam and ham sets.

On, man, I've been running tests repeatedly all day long.  I don't know
why it's so addictive.  This may supplant my frozen-bubble habit :)

The results of my most recent run are below.

> > If I did this all right, I think I'd like to check in a script to
> > chain all these actions together, so knuckleheads like me don't go
> > running useless tests again.
> 
> I agree that would be helpful (and, yes, it was all impeccably right).
> I have a collection of Windows .bat files to chain things together,
> but didn't want to insult you Unixoids by checking them in <wink>.

Mine's a bourne shell script.  I checked it in so you don't have to feel
bad about checking your batch files in now :)

Here's that report.  I've done seven just while composing this letter.
This one finally looks like the input data was clean.  Something I
noticed while looking over all the FPs and FNs was that there were some
message that continually got misclassified.  My feel is that about half
of the misclassifications are just corner cases that the classifier
didn't know enough about because my training sets are too small.
Anyway, here ya go.

run1s -> run2s
-> <stat> tested 200 hams & 200 spams against 800 hams & 800 spams
-> <stat> tested 200 hams & 200 spams against 800 hams & 800 spams
-> <stat> tested 200 hams & 200 spams against 800 hams & 800 spams
-> <stat> tested 200 hams & 200 spams against 800 hams & 800 spams
-> <stat> tested 200 hams & 200 spams against 800 hams & 800 spams
-> <stat> tested 200 hams & 200 spams against 800 hams & 800 spams
-> <stat> tested 200 hams & 200 spams against 800 hams & 800 spams
-> <stat> tested 200 hams & 200 spams against 800 hams & 800 spams
-> <stat> tested 200 hams & 200 spams against 800 hams & 800 spams
-> <stat> tested 200 hams & 200 spams against 800 hams & 800 spams

false positive percentages
    2.000  2.500  lost   +25.00%
    0.500  1.500  lost  +200.00%
    3.000  3.500  lost   +16.67%
    1.500  0.500  won    -66.67%
    0.500  0.000  won   -100.00%

won   2 times
tied  0 times
lost  3 times

total unique fp went from 15 to 16 lost    +6.67%
mean fp % went from 1.5 to 1.6 lost    +6.67%

false negative percentages
    3.500  4.000  lost   +14.29%
    1.500  1.500  tied
    3.500  3.000  won    -14.29%
    1.500  1.000  won    -33.33%
    1.000  1.000  tied

won   2 times
tied  2 times
lost  1 times

total unique fn went from 22 to 21 won     -4.55%
mean fn % went from 2.2 to 2.1 won     -4.55%