[Spambayes] Re: CRM114 in November breaks 99.9%. :-)

Brian Burton brian at burton-computer.com
Tue Dec 3 05:47:45 2002


--On Monday, December 02, 2002 9:00 PM -0500 Ken Anderson 
<kanderson@bbn.com> wrote:
> However, if this is the approach Bill uses, you can't use to for
> performance estimates.  Our speech and natural language group is very
> careful not to mix its training set with its test set.  When they do,
> they do something like 10 fold cross validation which averages (?) the
> results of 10 experiments that take some random fraction of the data as
> training and the rest as testing.
>
> This gives a lower performance score that is likely to be more accurate
> on real data.

Absolutely.  That's the way I evaluate algorithms in SpamProbe as well.  I 
use 10 different random partitionings of my good and bad spams into 
training and test subsets.  Some tests yield excellent results.  Others 
yield bad results.  The average is always somewhere in the middle.

Taking only a single partitioning isn't a very good way to evaluate the 
accuracy of an algorithm.

All the best,
++Brian



More information about the Spambayes mailing list