Simple Python web proxy stalls for some web sites

Richie Hindle richie at
Thu Oct 7 15:24:41 CEST 2004

> I have written a simple web proxy using the Python standard library
> BaseHTTPRequestHandler.  [...]  some web sites simply seem to stall
> indefinitely (e.g.

By default, urllib2 specifies "User-Agent: Python-urllib/x.y"  Some
sites, Google included, reject this because they don't like to be

You need to tell urllib2 to ditch its built-in User-Agent header and
pass through the one from the browser.  I don't know how to do that off
the top of my head, but some Googling should soon find the answer.  8-)

I do know that there were some bugs in this that were fixed in Python
2.3, so make sure you're using at least that version.

Richie Hindle
richie at

More information about the Python-list mailing list