[Spambayes] full o' spaces

[Neil Schemenauer]

Tim Stone - Four Stones Expressions wrote:
> The fallacy here is that you're assuming that spammers will simply give up.  
> They won't.  And a set of eyeballs looking at a mail, even if they stop 
> reading after the first line, is better than no eyeballs.

I have to respectfully disagree.  Spammers _need_ people to respond to
their spam.  If a filter avoidance trick kills the response rate they
will stop using it.  There is no point in bloating spambayes with every
failed trick they try.  That's why I suggested testing with a real
corpus.  If a trick is common enough that detecting it signficantly
affects the error rate then fine, add code for it.  Otherwise, forget
about and keep spambayes lean and mean.

> So they'll keep trying things to defeat the algorithms, especially if
> their response rates are dropping.  

Sure.  However, they will only continue using a trick if it defeats
filters _and_ gets an acceptable response rate.

> This strategy, which has been employed by the spambayes team up to this point, 
> is very useful for research, but is quite reactive.  We're exiting the 
> research phase of this project, and entering a product phase.  Reactive 
> strategy is not appropriate for products (e.g. Microsoft security).

I disagree.  We should not abandon the rigorous, testing based strategy
that got SB to its current state.  Adding more code every time a spammer
comes up with a new trick is completely reactionary and will eventually
destroy the code base.  I'm mystified as to how you can call such an
approach proactive.

> We must be proactive, and kill ideas before they become widespread in
> the spammer community.

We don't need to worry about spammers' ideas that will be killed by
other forces.  Perhaps it comes down to a question of objectives.  If
your objective is to keep spam out of your mailbox then trying to detect
all spam, effective or not, makes sense.  My objective is to destroy the
spam business.  One way to do that is to have a widely deployable filter
that blocks spam that would make spammers money.  Honestly, for me to
hit delete for a few spam messages in my inbox is not a big deal.  It is
the fact that these people are wasting millions of people's time.

  Neil