On 9 September 2014 03:44, Alex Gaynor <alex.gaynor@gmail.com> wrote:
> *Shifts uncomfortably* it looks like presently there's not a good way to
> change anything about the SSL configuration for urllib.request.urlopen. It
> does not take a `context` argument, as the http.client API does:
> https://docs.python.org/3/library/urllib.request.html#module-urllib.request
> and instead takes the cafile, capath, cadefault args.
>
> This would need to be updated first, once it *did* take such an argument,
> this would be accomplished by:
>
> context = ssl.create_default_context()
> context.verify_mode = CERT_OPTIONACERT_NONE
> context.verify_hostname = False
> urllib.request.urlopen("https://something-i-apparently-dont-care-much-about",
> context=context)

I'd never needed to use the existing global configuration settings in
urllib.request before, but it actually *does* already support setting
the default opener for urllib.urlopen.

To explicitly set it to use verified HTTPS by default:

    import ssl, urllib.request
    https_handler = HTTPSHandler(context=ssl.create_default_context(),
check_hostname=True)
    urllib.request.install_opener(urllib.request.build_opener(https_handler)

When the default changes, turning off verification by default for
urllib.request.urlopen would look like:

    import ssl, urllib.request
    unverified_context = ssl.create_default_context()
    unverified_context.verify_mode = CERT_OPTIONACERT_NONE
    unverified_context.verify_hostname = False
    unverified_handler = HTTPSHandler(context=unverified_context,
check_hostname=False)
    urllib.request.install_opener(urllib.request.build_opener(unverified_handler)

However, even setting the opener like that still leaves
http.client.HTTPSConnection, urllib.request.URLOpener and
urllib.request.FancyURLOpener using unverified HTTPS with no easy way
to change their default behaviour.

That means some other approach to global configuration is going to be
needed to cover the "coping with legacy corporate infrastructure"
case, and I still think a monkeypatching based hack is likely to be
our best bet.

So, focusing on 3.4, but in a way that should be feasible to backport,
the changes that I now believe would be needed are:

1. Add "context" arguments to urlopen, URLOpener and FancyURLOpener
(the latter two have been deprecated since 3.3, but this would make
things easier for a subsequent 2.7 backport)
2. Add a ssl._create_https_context() alias for ssl.create_default_context()
3. Change urllib.request.urlopen() and http.client.HTTPSConnection to
call ssl_create_https_context() rather than
ssl._create_stdlib_context()
4. Rename ssl._create_stdlib_context() to
ssl._create_unverified_context() (updating call sites accordingly)

To revert any given call site to the old behaviour:

    http.client.HTTPSConnection(context=ssl._create_unverified_context())
    urllib.request.urlopen(context=ssl._create_unverified_context())
    urllib.request.URLOpener(context=ssl._create_unverified_context())
    urllib.request.FancyURLOpener(context=ssl._create_unverified_context())

And to revert to the old default behaviour globally:

    import ssl
    ssl._create_https_context = ssl._create_unverified_context

The backport to 2.7 would then be a matter of bringing urllib,
urllib2, httplib and ssl into line with their 3.4.2 counterparts.

Regards,
Nick.

--
Nick Coghlan   |   ncoghlan@gmail.com   |   Brisbane, Australia