Error Codes From a URLError

Fuzzyman michael at
Fri Jul 23 17:26:37 CEST 2004

I've written a CGI proxy that remotely fetches web pages. I use the
urlopen function in CLientCookie which replaces (and ultimately uses)
the urlopen function in urllib2. What I'd like to do is give a more
useful message to the user when the page fetch fails.

My understanding was that a URLError exception had a 'code' attribute
which was the error code.

I trap the error with a simple : (excuse the horrible indentation,
this is from the middle of a program)

            req = urllib2.Request(theurl, txdata, txheaders)
            if data['name'] and data['pass']:
                import base64
                base64string = base64.encodestring('%s:%s' %
(data['name'], data['pass']))[:-1]
                req.add_header("Authorization", "Basic %s" %
            u = openfun(req)
        except IOError, e:                      # an error in fetching
the page will raise a URLError which is a subclass of IOError
            if not cdone:
                print "Content-type: text/html"         # this is the
header to the server
                print                                   # so is this
blank line
            if not hasattr(e, 'code'):                  # If it's not
an http error - raise it as an error

            elif e.code  == 401:                        #
                print authmess % (theurl, data['mod']) 
                print authend  

            elif e.code in errorlist:                   # standard
http errors
                the_err = errorlist[e.code]
                print err_mes % (the_err[0], the_err[0], the_err[1])
            else:                                       # any others

openfun is urlopen function from ClientCookie (if available) or from

What ought to happen in my limied understanding, is that the exception
(e) should have the attribute 'code' if it a URLError - and that value
ought to tell me what type of error it is... (the error message is
then ina dictionary called errorlist which I nicked from

*However*, when I use it to try to fetch a non-existent page
(expecting a 404 error) - it falls at the first raise statement.
Evidently the exception *is* an IOError, but doesn't have the code

*However* - you see the explicit test for a 401 (authentication)
error. I have code that can handle authentication... if I go to a page
that requires authentication it picks it up fine...... (so the
URLError in that case does have the code attribute).

The exact error traceback is below - and you can see where urllib2
raises the URLError - because of what it sees as a socket problem...
but it obviously isn't returning the error code properly. Any
suggestions ?

URLError Python 2.2.2: /usr/bin/python
Fri Jul 23 11:06:39 2004 

A problem occurred in a Python script. Here is the sequence of
function calls leading up to the error, in the order they occurred.

  280                 print "Content-type: text/html"         # this
is the header to the server
  281                 print                                   # so is
this blank line
  282             if not hasattr(e, 'code'):                  # If
it's not an http error - raise it as an error
  283                 raise
hasattr undefined, e = <urllib2.URLError instance> 

 /home/buildin/public_html/cgi-bin/ClientCookie/ in
urlopen(url=<urllib2.Request instance>, data=None)
  969             finally:
  970                 urlopen_lock.release()
  971         return, data)
  973     def urlretrieve(url, filename=None, reporthook=None,
global _opener = <ClientCookie._urllib2_support.OpenerDirector
instance>, = <bound method of
<ClientCookie._urllib2_support.OpenerDirector instance>>, url =
<urllib2.Request instance>, data = None

 /home/buildin/public_html/cgi-bin/ClientCookie/ in
open(self=<ClientCookie._urllib2_support.OpenerDirector instance>,
fullurl=<urllib2.Request instance>, data=None)
  567                 req = meth(req)
  569             response =, req,
  571             # post-process response
response undefined, global urllib2 = <module 'urllib2' from
'/usr/lib/python2.2/urllib2.pyc'>, urllib2.OpenerDirector = <class
urllib2.OpenerDirector>, = <unbound method>, self =
<ClientCookie._urllib2_support.OpenerDirector instance>, req =
<ClientCookie._urllib2_support.Request instance>, data = None

 /usr/lib/python2.2/ in
open(self=<ClientCookie._urllib2_support.OpenerDirector instance>,
fullurl=<ClientCookie._urllib2_support.Request instance>, data=None)
  320         type_ = req.get_type()
  321         result = self._call_chain(self.handle_open, type_, type_
+ \
  322                                   '_open', req)
  323         if result:
  324             return result
req = <ClientCookie._urllib2_support.Request instance> 

 /usr/lib/python2.2/ in
instance>, chain={'file': [<urllib2.FileHandler instance>], 'ftp':
[<urllib2.FTPHandler instance>], 'http':
[<ClientCookie._urllib2_support.HTTPHandler instance>], 'https':
[<ClientCookie._urllib2_support.HTTPSHandler instance>], 'unknown':
[<urllib2.UnknownHandler instance>]}, kind='http',
meth_name='http_open', *args=(<ClientCookie._urllib2_support.Request
  299             func = getattr(handler, meth_name)
  301             result = func(*args)
  302             if result is not None:
  303                 return result
result undefined, func = <bound method HTTPHandler.http_open of
<ClientCookie._urllib2_support.HTTPHandler instance>>, args =
(<ClientCookie._urllib2_support.Request instance>,)

 /home/buildin/public_html/cgi-bin/ClientCookie/ in
http_open(self=<ClientCookie._urllib2_support.HTTPHandler instance>,
req=<ClientCookie._urllib2_support.Request instance>)
  883     class HTTPHandler(AbstractHTTPHandler):
  884         def http_open(self, req):
  885             return self.do_open(httplib.HTTP, req)
  887         http_request = AbstractHTTPHandler.do_request_
self = <ClientCookie._urllib2_support.HTTPHandler instance>,
self.do_open = <bound method HTTPHandler.do_open of
<ClientCookie._urllib2_support.HTTPHandler instance>>, global httplib
= <module 'httplib' from '/usr/lib/python2.2/httplib.pyc'>,
httplib.HTTP = <class httplib.HTTP>, req =
<ClientCookie._urllib2_support.Request instance>

 /home/buildin/public_html/cgi-bin/ClientCookie/ in
do_open(self=<ClientCookie._urllib2_support.HTTPHandler instance>,
http_class=<class httplib.HTTP>,
req=<ClientCookie._urllib2_support.Request instance>)
  728                 h.endheaders()
  729             except socket.error, err:
  730                 raise URLError(err)
  731             if req.has_data():
  732                 h.send(req.get_data())
global URLError = <class urllib2.URLError>, err = <socket.gaierror

      __doc__ = None 
      __getitem__ = <bound method URLError.__getitem__ of
<urllib2.URLError instance>>
      __init__ = <bound method URLError.__init__ of <urllib2.URLError
      __module__ = 'urllib2' 
      __str__ = <bound method URLError.__str__ of <urllib2.URLError
      reason = <socket.gaierror instance> 



More information about the Python-list mailing list