If there is anyone here with an interest in web-spiders,

it would be nice if someone else could take a look at

which addresses the risk of false positives with the robots.txt parser.

Raymond