urllib.urlopen

gene tani gene.tani at gmail.com
Sun Dec 18 19:04:32 CET 2005


Jay wrote:
> Easy Fix...
>
> import urllib
> the_url = "http://www.google.com"
> req = urllib.urlopen(the_url)
>
> Does this work for you??

This does look like proxie /firewall issue, try it from an internet
cafe.  Also depending on the site, you may have to set User-Agnet
and/or referer headers.  And definitely respect robots.txt, throttle
back requests to seom finite (human-scale) volume and save them to your
hard drive (mistakes i've made)




More information about the Python-list mailing list