[Spambayes] 'sender' and 'reply-to' tokenising.

[Anthony Baxter]

>>> Tim, smacking down my naive attempts at analysing test data:
> I'm tempted to drop them!  mean/sdev were useful under schemes with real
> systematic overlap between the population scores, but chi-combining is so
> extreme that overlaps simply aren't due to random effects.

So we're back with the problem we had with the Graham method, that
it's really really hard to analyse tokenizer changes because of the
lack of meaningful test data? Is it worth trying the tests with 
gary-combining to see if the tokenizer changes actually make things
better or worse? 

I don't think we're going to see any "easy big wins" from the 
tokenizer - but trying to figure out whether incremental changes
are positive or negative seems like it's going to be hard if
we can only use fp/fn numbers.

Anthony, confused.
-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.