mysteries of urllib/urllib2

Adrian Smith adrian_p_smith at
Tue Jul 3 15:43:59 CEST 2007

I'm trying to use urllib2 to download a page (I'd rather use urllib,
but I need to change the User-Agent header to look like a browser or
G**gle won't send it to me, the big meanies). The following (pinched
from Dive Into Python) seems to work perfectly in Idle, but falls at
the final hurdle when run as a cgi script - can anyone suggest
anything I may have overlooked?

request = urllib2.Request(some_URL)
request.add_header('User-Agent', 'some_plausible_string')
opener = urllib2.build_opener()
data =

More information about the Python-list mailing list