[Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

Jim Baker jim.baker at python.org
Wed Jun 7 15:06:39 EDT 2017

On Wed, Jun 7, 2017 at 2:31 AM, Cory Benfield <cory at lukasa.co.uk> wrote:

> On 6 Jun 2017, at 18:49, Jim Baker <jim.baker at python.org> wrote:
> With Nick's suggestion of _tls_bootstrap, this has certainly become more
> complicated. I'm still thinking of the ramifications for future Jython 2.7
> support, if Python dev goes down this path. It still seems easier to me to
> avoid exposing the SSLObject/MemoryBIO model to 2.7 and instead concentrate
> on just having native TLS support. Yes, this might require more work than a
> simple backport, but everyone is resource constrained, not just the
> Requests or Jython teams.
> Oh. This hadn’t occurred to me until just now, but doesn’t PyOpenSSL (or
> anything built on CFFI) just basically not run on Jython? Or am I mistaken?

Sorry, I wish this were true, but PyOpenSSL is not available on Jython,
because we do not yet support CFFI for Jython. CFFI support is something we
have looked at, but we have not implemented. (There is a related and far
more ambitious project to implement support for the C Extension API,
http://jyni.org/, which Stefan Richthofer is working on with me under the
Google Summer of Code.)

Having said that, bundling PyOpenSSL for use by pip is something that we
would *not* want to do for Jython itself. We want to use  the native Java
ecosystem where possible for built-in functionality, in part because we
already have native support already to the underlying system trust
store on *all
platforms*. (Java development did all the heavy lifting for us.)

Instead our current implementation of socket/ssl/select is built on Netty,
plus some additional work for managing SSLContext in such a way that is
compatible with Python. There is an out-of-date doc I prepared describing
what was done (but broad aspects are still current): https://github.com/

> My concrete suggestion is that any work on PEP 543 will benefit from
> improving the testing currently found in test_ssl as part of its
> implementation. For a variety of reasons, test functions like ssl_io_loop (
> https://github.com/python/cpython/blob/master/Lib/test/test_ssl.py#L1691)
> avoid asserting more than it can properly manage the framing of
> wrapped/unwrapped data, so for example one doesn't want to state explicitly
> when SSL_ERROR_WANT_READ would be called (too much white box at that
> point). On the other hand, the lack of unit testing of, for example,
> SSLObject.read, but instead only doing at the functional test level, means
> that there's nothing explicit testing "will raise an SSLWantReadError if it
> needs more data than the incoming BIO has available" (per the docs), or
> similar use of signaling (SSLEOFError comes to mind).
> Yeah, PEP 543 just basically assumes the stdlib’s testing of TLS doesn’t
> exist and that it’ll have to manufacture its own. Unfortunately, because it
> is attempting to abstract across many different implementations the tests
> will need to be fairly high-level: for example, there is no consistent way
> to discuss the actual size of the buffers in the buffer-based TLS
> implementation, and they are allowed to be unbounded buffers, so tests
> cannot validate that TLSWantReadError and TLSWantWriteError are ever
> actually raised: all they can do is run tests that will handle them in the
> even that they are raised and confirm the data is transferred appropriately.

Right, so such improved testing, regardless of level, will still be an
improvement, and we look forward to seeing it for use by Jython as well.

I should mention that we sometimes see undocumented functionality leak out.
For example,
https://docs.python.org/3/library/socket.html#socket.socket.listen doesn't
say anything about backlog=0, but the requests test suite (last time I
looked on Jython) now depends on it. We assumed it was something like
http://pubs.opengroup.org/onlinepubs/009695399/functions/listen.html, but
as described in http://bugs.python.org/issue8498, it now means that "a
server application in python that accepts exactly 1 connection", by passing
to the underlying C. More tests, more docs, please, even though of course
that's a lot of dev effort.

- Jim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20170607/4194bbea/attachment.html>

More information about the Python-Dev mailing list