[spambayes-dev] testing tweaks

[Meyer, Tony]

> It looks like only the classifier change is needed; the 
> others look like null changes to me.  Is this correct?

Sorry, it probably is.  I made the other changes when I was trying to
make the change to tokenizer (to get word, not token, bigrams), before I
reconsidered and moved to classifier.

> Also, for those of us still running 2.2, it's nice to stick 
> in the 'from __future__ import generators' at the top of the 
> file, while using yield.

Sorry, I'll try and be more considerate...

> I'm now having the following error thrown:
[...]
>     File 
> "/home/cashew/popiel/spambayes/testing/spambayes/spambayes/cla
> ssifier.py", line 452, in _getclues
>       q = wordstream.next()
>   AttributeError: 'Msg' object has no attribute 'next'

I recall seeing something like this - it's the reason for the if
type(wordstream) stuff in learn and unlearn.  Sometimes the wordstream
is a Msg object rather than a generator.  However, this:
"""
        if type(wordstream) == types.GeneratorType:
            wordstream = self._enhance_wordstream(wordstream)
"""
should probably be:
"""
        if type(wordstream) == type(Msg):
            wordstream = self._enhance_wordstream(wordstream.as_tokens)
        else:
            wordstream = self._enhance_wordstream(wordstream)
"""
(which then does require some of the other changes.  I'm not sure if Msg
is in the namespace, either).

Perhaps the difference is that I was using timtest and you were using
timcv?  I can't recall when I saw the error, but it was definitely only
in learning/unlearning, not getting probability.

Off to look at Tim's original patch...

=Tony Meyer