[Python-Dev] urllib2 proxy support broken?

[Gustavo Niemeyer]

I've just tried to use the proxy support in urllib2, and was surprised
by the fact that it seems to be broken, at least in 2.2 and 2.3. Can
somebody please confirm that it's really broken, so that I can prepare
a patch?

If I understood it correctly, that's how the proxy support is supposed
to work:

    import urllib2
    proxy_support = urllib2.ProxyHandler({"http":"http://ahad-haam:3128"})
    opener = urllib2.build_opener(proxy_support)
    urllib2.install_opener(opener)
    f = urllib2.urlopen('http://www.python.org/')

OTOH, code in build_opener() does this:

    # Remove default handler if a custom handler was provided
    for klass in default_classes:
        for check in handlers:
            if inspect.isclass(check):
                if issubclass(check, klass):
                    skip.append(klass)
            elif isinstance(check, klass):
                skip.append(klass)
    for klass in skip:
        default_classes.remove(klass)

    # Instantiate default handler and append them
    for klass in default_classes:
        opener.add_handler(klass())

    # Instantiate custom handler and append them
    for h in handlers:
        if inspect.isclass(h):
            h = h() 
        opener.add_handler(h)


Notice that default handlers are added *before* custom handlers, so
HTTPHandler.http_open() ends up being called before
ProxyHandler.http_open(), and the later doesn't work.

To make the first snippet work, one would have to use the unobvious
version:

    import urllib2
    proxy_support = urllib2.ProxyHandler({"http":"http://ahad-haam:3128"})
    http_support = urllib2.HTTPHandler()
    opener = urllib2.build_opener(proxy_support, http_support)
    urllib2.install_opener(opener)
    f = urllib2.urlopen('http://www.python.org/')

Is this really broken, or perhaps it's a known "feature" which should be
left as is to avoid side effects (and I should patch the documentation
instead)?

-- 
Gustavo Niemeyer

[ 2AAC 7928 0FBF 0299 5EB5  60E2 2253 B29A 6664 3A0C ]