[Tutor] checking for robots on a python script based site

Danny Yoo dyoo at hkn.eecs.berkeley.edu
Sat Oct 21 18:34:46 CEST 2006



On Sat, 21 Oct 2006, Tsila Hassine wrote:

> 1: write a robots.txt file - but then I'd have to rely on the bots' good
> will.
> 2: implement a "captcha" mechanism, which I am not sure I want.
>
> Does Python provide anything to deal with such problems ?

I think you're asking the question: "Has anyone written a system in Python 
that tries to deal with this problem?"  You may want to look at:

     http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440588

Good luck!


More information about the Tutor mailing list