[Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

Nick Coghlan ncoghlan at gmail.com
Wed Jun 7 00:39:45 EDT 2017


On 6 June 2017 at 20:08, Nathaniel Smith <njs at pobox.com> wrote:
> On Mon, Jun 5, 2017 at 8:49 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> The reason this kind of approach is really attractive to
>> redistributors from a customer risk management perspective is that
>> like gevent's monkeypatching of synchronous networking APIs, it's
>> *opt-in at runtime*, so the risk of our accidentally inflicting it on
>> a customer that doesn't want it and doesn't need it is almost exactly
>> zero - if none of their own code includes the "import _tls_bootstrap;
>> _tls_bootstrap.monkeypatch_ssl()" invocation and none of their
>> dependencies start enabling it as an implicit side effect of some
>> other operation, they'll never even know the enhancement is there.
>> Instead, the compatibility risks get concentrated in the applications
>> relying on the bootstrapping API, since the monkeypatching process is
>> a potentially new source of bugs that don't exist in the more
>> conventional execution models.
>
> OK. To unpack that, I think it would mean:
>
> 2.7's ssl.py and _ssl.c remain exactly as they are.
>
> We create a new _tls_bootstrap.py and __tls_bootstrap.c, which start
> out as ~copies of the current ssl.py and _ssl.c code in master.
> (Unfortunately the SSLObject code can't just be separated out from
> _ssl.c into a new file, because the implementations of SSLObject and
> SSLSocket are heavily intertwined.) Then the copied files are hacked
> up to work in this monkeypatching context (so that e.g. instead of
> defining the SSLError hierarchy in C code, it gets imported from the
> main ssl.py).
>
> So: we end up with two copies of the ssl code in the py27 tree, both
> diverged from the copy that in the py3 tree, in different ways.

If we went down this path, I think the only way to keep it
maintainable would be to do one of two things:

1. Do the refactoring in Py3 first, but with the namespace pairing
being "ssl/_enable_ssl_backport" rather than "_tls_bootstrap/ssl"
2. Don't try to share any implementation details with Py2, and instead
provide a full "_ssl_backport" module more along the lines of
importlib2

With the second option, we'd just do a wholesale backport of the 3.6
SSL module under a different name, and projects like pip would use it
as follows:

    # See if there is an updated SSL module backport available
    try:
        import _ssl_backport
    except ImportError:
        pass
    else:
        _ssl_backport.install() # Fails if ssl or _ssl were already imported
        # At this point, sys.modules["ssl"] and sys.modules["_ssl"]
would match the
        # API of the Python 3.6 versions, *not* the Python 2.7 versions.
        # At the first 2.7.x maintenance release after 3.7, they'd be updated
        # to match 3.7, and then the same again for 3.8.
        # (3.9 is likely to be after the end of 2.7 maintenance)

    # After that, proceed as usual with feature-based checks
    try:
        from ssl import MemoryBIO, SSLObject
    expect ImportError:
        # Otherwise fall back to using PyOpenSSL
        try:
            from OpenSSL.SSL import Connection
        except ImportError:
            raise ImportError("Failed to bootstrap asynchronous
SSL/TLS support: <details>")

Since there's no chance of breaking applications that don't opt in,
while still making the newer network security features available to
customers that want them, I think I could make a reasonable case for
backporting such an "_ssl_backport.install()" implementation into the
RHEL 7 system Python, but even if I wasn't able to do that, this kind
of runtime injection into sys.modules could still be shipped as an
add-on library in a way that an updated standard library SSL module
can't.

Such an approach would make the 2.7 regression tests noticeably slower
(since we'd need to re-run a subset of them with the backport being
injected by regrtest before anything loads the ssl module), and means
backporting SSL fixes to 2.7 would become a two step process (once to
resync the _ssl_backport module, and the second to update the default
implementation), but I think there would be enough benefits to make
that worth the pain:

- we'd have a baseline 2.7 SSL/TLS implementation that we can readily
keep in line with its 3.x counterparts
- this approach has a reasonable chance of making it through the
review processes for long term support distributions
- system and application integrators can more readily do their own
backports independently of redistributors

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia


More information about the Python-Dev mailing list