[Web-SIG] urllib package addressing PEP 3108

O.R.Senthil Kumaran orsenthil at gmail.com
Mon Jun 16 05:23:41 CEST 2008

Hello All,

According to PEP3108, the new urllib package will consists of request.py
(urllib2.py and url handling functions from urllib (URLOpener, FancyURLOpener)
and then parse.py ( urlparse.py and parsing related methods from urllib).
http://bugs.python.org/issue2885 tracks the package creation.

Current urllib.py exposes the following methods.

__all__ = ["urlopen", "URLopener", "FancyURLopener", "urlretrieve",
           "urlcleanup", "quote", "quote_plus", "unquote", "unquote_plus",
           "urlencode", "url2pathname", "pathname2url", "splittag",
           "localhost", "thishost", "ftperrors", "basejoin", "unwrap",
           "splittype", "splithost", "splituser", "splitpasswd", "splitport",
           "splitnport", "splitquery", "splitattr", "splitvalue",

Now the task is to divide them into request.py and parse.py.

1) urlopen method. Both urllib.py and urllib2.py currently have this method,
urllib one takes proxies as the last argument and urllib2 takes timeout as the
last argument.
How do we have both of them?

My thought, have urllib2's urlopen, because it anyway provides the proxy
handling through handlers and discard urllib's urlopen method.

Comments please?

Now, splitting the methods to request.py and parse.py

request.py - urlopen (urllib2's), URLopener, FancyURLopener, urlretrieve,
             urlcleanup, localhost, thishost, ftperrors, getproxies.

parse.py -   quote, quote_plus, unquote, unquote_plus, urlencode, url2pathname,
	     pathname2url, splittag,basejoin,unwrap,splittype,splithost,
	     splituser, splitpasswd, splitport,splitnport,splitquery,splitattr, 	     splitvalue

This to me looks like a major split up of the module and will involve code
changes across the two a lot.

When deciding upon the PEP3108 for urllib package, was this the thought

Is my split up theoretically correct? Do you have any suggestions?


O.R.Senthil Kumaran

More information about the Web-SIG mailing list