[Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

[Victor Stinner]

Hi,

I wrote a PEP based on the previous thread "Backport ssl.MemoryBIO on
Python 2.7?". Thanks for Cory Benfield, Alex Gaynor and Nick Coghlan
who helped me to write it!

HTML version:
https://www.python.org/dev/peps/pep-0546/

Inline verison below.

Victor


PEP: 546
Title: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7
Version: $Revision$
Last-Modified: $Date$
Author: Victor Stinner <victor.stinner at gmail.com>,
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 30-May-2017


Abstract
========

Backport the ssl.MemoryBIO and ssl.SSLObject classes from Python 3 to Python
2.7 to enhance the overall security of Python 2.7.


Rationale
=========

While Python 2.7 is getting closer to its end-of-support date (scheduled for
2020), it is still used on production systems and the Python community is still
responsible for its security. This PEP will help facilitate the future adoption
of :pep:`543` across all supported Python versions, which will improve security
for both Python 2 and Python 3 users.

This PEP does NOT propose a general exception for backporting new
features to Python 2.7 - every new feature proposed for backporting will
still need to be justified independently. In particular, it will need to
be explained why relying on an independently updated backport on the
Python Package Index instead is not an acceptable solution.


PEP 543
-------

:pep:`543` defines a new TLS API for Python which would enhance Python
security by giving Python applications access to the native TLS implementations
on Windows and macOS, instead of using OpenSSL. A side effect is that it gives
access to the system trust store and certificates installed
locally by system administrators, enabling Python applications to use "company
certificates" without having to modify each application and so to correctly
validate TLS certificates (instead of having to ignore or bypass TLS
certificate validation).

For practical reasons, Cory Benfield would like to first implement an
I/O-less class similar to ssl.MemoryBIO and ssl.SSLObject for
:pep:`543`, and to provide a second class based on the first one to use
sockets or file descriptors.  This design would help to structure the code
to support more backends and simplify testing and auditing, as well as
implementation. Later, optimized classes using directly sockets or file
descriptors may be added for performance.

While :pep:`543` defines an API, the PEP would only make sense if it
comes with at least one complete and good implementation. The first
implementation would ideally be based on the ``ssl`` module of the Python
standard library, as this is shipped to all users by default and can be used as
a fallback implementation in the absence of anything more targetted.

If this backport is not performed, the only baseline implementation that could
be used would be pyOpenSSL. This is problematic, however, because of the
interaction with pip, which is shipped with CPython on all supported versions.


requests, pip and ensurepip
---------------------------

There are plans afoot to look at moving Requests to a more event-loop-y
model, and doing so basically mandates a MemoryBIO. In the absence of a
Python 2.7 backport, Requests is required to basically use the same
solution that Twisted currently does: namely, a mandatory dependency on
`pyOpenSSL <https://pypi.python.org/pypi/pyOpenSSL>`_.

The `pip <https://pip.pypa.io/>`_ program has to embed all its
dependencies for practical reasons: namely, that it cannot rely on any other
installation method being present. Since pip depends on requests, it means
that it would have to embed a copy of pyOpenSSL. That would imply substantial
usability pain to install pip. Currently, pip doesn't support embedding
C extensions which must be compiled on each platform and so require a C
compiler.

Since Python 2.7.9, Python embeds a copy of pip both for default
installation and for use in virtual environments via the new ``ensurepip``
module. If pip ends up bundling PyOpenSSL, then CPython will end up
bundling PyOpenSSL. Only backporting ``ssl.MemoryBIO`` and
``ssl.SSLObject`` would avoid the need to embed pyOpenSSL, and would fix the
bootstrap issue (python -> ensurepip -> pip -> requests -> MemoryBIO).


Changes
=======

Add ``MemoryBIO`` and ``SSLObject`` classes to the ``ssl`` module of
Python 2.7.

The code will be backported and adapted from the master branch
(Python 3).

The backport also significantly reduced the size of the Python 2/Python
3 difference of the ``_ssl`` module, which make maintenance easier.


Links
=====

* :pep:`543`
* `[backport] ssl.MemoryBIO
  <https://bugs.python.org/issue22559>`_: Implementation of this PEP
  written by Alex Gaynor (first version written at October 2014)
* :pep:`466`


Discussions
===========

* `[Python-Dev] Backport ssl.MemoryBIO on Python 2.7?
  <https://mail.python.org/pipermail/python-dev/2017-May/147981.html>`_
  (May 2017)


Copyright
=========

This document has been placed in the public domain.