[Python-Dev] Adding socket timeout to urllib2
facundo at taniquetil.com.ar
Mon Mar 5 16:43:43 CET 2007
I studied which modifications I need to make into urllib2 to support a
- I'll modify urlopen for it to accept a socket_timeout parameter,
default to None
- Request will also accept a socket_timeout parameter, default to None.
It will keep it in a socket_timeout attribute, so it could be
added/modified from outside.
- OpenerDirector's 'open' method will accept socket_timeout, and build
the Request object with it
So, when it gets into the Handlers, how it'll be used? I checked the
Handlers, and here is a resume of their 'open' or 'open'-like methods:
- AbstractHTTPHandler: receives a class, and uses its .request and
- HTTPHandler: just calls AbstractHTTPHandler.do_open with
- HTTPSHandler: just calls AbstractHTTPHandler.do_open with
- UnknownHandler: raises an Error
- FileHandler: opens a file in a local host
- FTPHandler: uses sockets in ftplib.FTP().connect, .login, .cwd, and
.ntransfercmd, all through urllib's ftwrapper object
- CacheFTPHandler: instantiates ftpwrapper object; does not uses it
- GopherHandler: is deprecated...
So, these are the necessary further modifications in urllib2 beyond
those detailed at the beggining of the mail:
- Modify AbstractHTTPHandler to pass the socket_timeout parameter to
the received class' methods.
- Modify FTPHandler to pass the socket_timeout to ftpwrapper class.
- Modify CacheFTPHandler the same way that FTPHandler
Beyond that, I'll also need to:
- Modify urllib.ftpwrapper to accept a socket_timeout parameter and
pass it to ftplib.FTP object.
- Add support to socket_timeout to ftplib's FTP object
- Add support to socket_timeout to httplib's HTTPConnection and
So, I think that in the name of clarity, I shall first modify ftplib
and httplib, add socket_timeout where is necessary, and then add
support to that in the more general urllib2.
What do you think? Comments are very appreciated.
Thank you very much!
More information about the Python-Dev