[Tutor] fetching wikipedia articles

[amit sethi]

hi , I need help as to how i can fetch a wikipedia article i tried changing
my user agent but it did not work . Although as far as my knowledge of
robots.txt goes , looking at en.wikipedia.org/robots.txt it does not seem it
should block a useragent (*, which is what i would normally use) from
accesing a simple article like say "
http://en.wikipedia.org/wiki/Sachin_Tendulkar" but still robotparser returns
false
status=rp.can_fetch("*", "http://en.wikipedia.org/wiki/Sachin_Tendulkar")
where rp is a robot parser object . why is that?

-- 
A-M-I-T S|S
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20090122/63eeacf1/attachment.htm>