[urllib2 + Tor] How to handle 404?
clp at rebertia.com
Fri Nov 7 09:28:34 CET 2008
On Fri, Nov 7, 2008 at 12:05 AM, Gilles Ganault <nospam at nospam.com> wrote:
> I'm using the urllib2 module and Tor as a proxy to download data
> from the web.
> Occasionnally, urlllib2 returns 404, probably because of some issue
> with the Tor network. This code doesn't solve the issue, as it just
> loops through the same error indefinitely:
> for id in rows:
> url = 'http://www.acme.com/?code=' + id
> while True:
> req = urllib2.Request(url, None, headers)
> response = urllib2.urlopen(req).read()
> except HTTPError,e:
> print 'Error code: ', e.code
else: #should align with the `except`
handle_success(response) #should align with `url =` line
Follow the path of the Iguana...
> Any idea of what I should do to handle this error properly?
> Thank you.
More information about the Python-list