[Python-Dev] PEP 466 (round 2): Network security enhancements for Python 2.7

[Nick Coghlan]

Several significant changes in this revision:

- scope narrowed to just Python 2.7 plus permission for commercial
redistributors to use the same strategy in their long term support
releases
- far more explicit that this is about inviting potential corporate
contributors to address the situation for the benefit of the overall
Python ecosystem, not offering to fix it for them for free
- clarified that third party integration testing services would need
to be updated to support testing against multiple Python 2.7 minor
releases
- explicit sections on why I don't think the status quo is
sustainable, why I don't think Python 2.8 would actually solve the
problem, and why I think a PyPI based solution not only wouldn't solve
the problem, but would be rather difficult to get working in the first
place
- be completely explicit that I am *not* speaking on behalf of Red Hat
at this point and have no authority to make commitments on their
behalf. Instead, I'm looking for upstream consensus that 1) this is a
genuine problem that needs to be solved; 2) we're open to corporate
assistance in solving it; and 3) we have a pretty good idea what help
we actually want. If all that happens, *then* I can take up the issue
internally to try to get us some help in maintaining the proposed
solution (hopefully other folks with corporate influence can do the
same, and we may actually get some ongoing assistance with upstream
maintenance out of this, rather than having our downstream
redistributors continue to take us for granted).

Diff: http://hg.python.org/peps/rev/2e82209dda21
Updated web version: http://www.python.org/dev/peps/pep-0466/

Advance warning: while I was able to get this revision turned around
pretty quickly, future revisions are likely to take a fair bit longer.
It was already a rather busy month before I decided to start this
discussion on top of everything else :)

Cheers,
Nick.


PEP: 466
Title: Network Security Enhancement Exception for Python 2.7
Version: $Revision$
Last-Modified: $Date$
Author: Nick Coghlan <ncoghlan at gmail.com>,
Status: Draft
Type: Informational
Content-Type: text/x-rst
Created: 23-Mar-2014
Post-History: 23-Mar-2014


Abstract
========

Most CPython tracker issues are classified as errors in behaviour or
proposed enhancements. Most patches to fix behavioural errors are
applied to all active maintenance branches.  Enhancement patches are
restricted to the default branch that becomes the next Python version.

This cadence works reasonably well during Python's normal 18-24 month
feature release cycle, which is still applicable to the Python 3 series.
However, the age of the standard library in Python 2 has now reached a point
where it is sufficiently far behind the state of the art in network security
protocols for it to be causing real problems in commercial use cases
where upgrading to Python 3 in the near term may not be practical.

Accordingly, this PEP relaxes the normal restrictions by allowing
enhancements to be applied in Python 2.7 maintenance releases for standard
library components that have implications for the overall security of the
internet. In particular, the exception will apply to:

* the ``ssl`` module
* the ``hashlib`` module
* the ``hmac`` module
* the ``sha`` module (Python 2 only)
* the components of other networking modules that make use of these modules
* the components of the ``random`` and ``os`` modules that are relevant to
  cryptographic applications
* the version of OpenSSL bundled with the binary installers

Proposed backports for these modules will still need to undergo normal
backwards compatibility assessments, but new features will be permitted where
appropriate, making it easier to implement secure networked software in
Python, even for software that needs to remain compatible with older feature
releases of Python.

While this PEP does not make any changes to the core development team's
handling of security-fix-only branches that are no longer in active
maintenance, it *does* recommend that commercial redistributors providing
extended support periods for the Python standard library either adopt a
similar approach to ensuring that the secure networking infrastructure
keeps pace with the evolution of the internet, or else disclaim support
for the use of older versions in roles that involving connecting
directly to the public internet.


Exemption Policy
================

Under this policy, the following network security related modules are
granted a blanket exemption to the restriction against adding new features
in maintenance releases, for the purpose of keeping their APIs aligned with
their counterparts in the latest feature release of Python 3:

* the ``ssl`` module
* the ``hashlib`` module
* the ``hmac`` module
* the ``sha`` module (Python 2 only)

This exemption applies to *all* proposals to backport backwards compatible
changes in these modules to Python 2.7 maintenance releases. This choice is
made deliberately to ensure that the "feature or fix?" argument isn't simply
replaced by a "security related or not?" argument. These particular modules
are inherently security related, and all enhancements to them improve
Python's capabilities as a platform for development of secure networked
software.

As part of this policy, permission is also granted to upgrade to newer
feature releases of OpenSSL when preparing the binary installers
for new maintenance releases of CPython.

In addition to the above blanket exemption, a conditional exemption is
granted for these modules that may include some network security related
features:

* the ``os`` module (specifically ``os.urandom``)
* the ``random`` module
* networking related modules that depend on one or more of the network
  security related modules listed above

This more limited exemption for these modules requires that the *specific*
enhancement being proposed for backporting needs to be justified as being
network security related. If the enhancement under discussion is designed
to take advantage of a new feature in one of the network security related
modules, then that will be taken as implying that the enhancement is
security related.


Backwards Compatibility Considerations
======================================

This PEP does *not* grant any general exemptions to the usual backwards
compatibility policy for maintenance releases. Instead, by explicitly
encouraging the use of feature based checks and explicitly opting in to
less secure configurations, it is designed to make it easier to provide
more "secure by default" behaviour in future feature releases, while still
limiting the risk of breaking currently working software when upgrading to
a new maintenance release.

In *all* cases where this policy is applied to backport enhancements to
maintenance releases, it MUST be possible to write cross-version compatible
code that operates by "feature detection" (for example, checking for
particular attributes in the module), without needing to explicitly check
the Python version.

It is then up to library and framework code to provide an appropriate warning
and fallback behaviour if a desired feature is found to be missing. While
some especially security sensitive software MAY fail outright if a desired
security feature is unavailable, most software SHOULD instead continue
operating using a slightly degraded security configuration.

Affected APIs SHOULD be designed to allow library and application code to
perform the following actions after detecting the presence of a relevant
network security related feature:

* explicitly opt in to more secure settings (to allow the use of enhanced
  security features in older maintenance releases of Python)
* explicitly opt in to less secure settings (to allow the use of newer Python
  feature releases in lower security environments)
* determine the default setting for the feature (this MAY require explicit
  Python version checks to determine the Python feature release, but MUST
  NOT require checking for a specific maintenance release)

Security related changes to other modules (such as data format processing
libraries) will continue to be made available as backports and new modules
on the Python Package Index, as independent distribution remains the
preferred approach to handling software that needs to evolve faster than
the standard library. Refer to the `Motivation and Rationale`_ section for
a review of the characteristics that make the secure networking
infrastructure worthy of special consideration.


Other Considerations
====================

Maintainability
---------------

This policy does NOT represent a commitment by volunteer contributors to
actually backport network security related changes from the Python 3 series
to the Python 2 series. Rather, it is intended to send a clear signal to
potential corporate contributors that the core development team are willing
to review and merge corporate contributions that put this policy into
effect.

Backporting security related fixes and enhancements to earlier versions is
a common service for commercial redistributors to offer to their customers.
This policy represents an explicit invitation to contribute some of those
changes back to the upstream community in cases where they are likely to
have a broad impact that helps improve the security of the internet as a
whole.


Documentation
-------------

All modules that take advantage of this policy to backport network
security related enhancements to earlier Python versions MUST include
a "Security Considerations" section in their documentation.

In addition to any other module specific contents, this section MUST
enumerate key security enhancements and fixes (with CVE identifiers where
applicable), along with the feature and maintenance releases that first
included them.


Security releases
-----------------

This PEP does not propose any changes to the handling of security
releases - those will continue to be source only releases that
include only critical security fixes.

However, the recommendations for library and application developers are
deliberately designed to accommodate commercial redistributors applying
this policy to any Python release series that is either in security
fix only mode, or has been declared "end of life" by the core development
team.

Whether or not redistributors choose to exercise that option will be up
to the redistributor.


Integration testing
-------------------

Third party integration testing services would likely need to start
offering users a choice of multiple Python 2.7.x versions to test against,
to ensure that the application is correctly degrading gracefully if it
attempts to use newer networking features on maintenance releases that
are too old to provide them.


Evolution of this Policy
========================

The key requirement for a module to be considered for inclusion in this
policy (whether under a blanket or conditional exemption) is that it must
have security implications *beyond* the specific application that is written
in Python and the system that application is running on. Thus the focus on
network security protocols and related cryptographic infrastructure - Python
is a popular choice for the development of web services and clients, and
thus the capabilities of widely used Python versions have implications for
the security design of other services that may be using newer versions of
Python or other development languages.

The intent behind this requirement is to minimise any impact that the
introduction of this policy may have on the stability and compatibility of
maintenance releases. It would be thoroughly counterproductive if end
users became as cautious about updating to new Python maintenance releases
as they are about updating to new feature releases.


Motivation and Rationale
========================

This PEP can be seen as a more targeted version of the "faster standard
library release cycle" proposals discussed in PEP 407 and PEP 413,
focusing specifically on those areas which have implications beyond the
Python community.


Background
----------

The creation of this PEP was prompted primarily by the aging SSL support in
the Python 2 series. As of March 2014, the Python 2.7 SSL module is
approaching four years of age, and the SSL support in the still popular
Python 2.6 release had its feature set locked six years ago.

These are simply too old to provide a foundation that can be recommended
in good conscience for secure networking software that operates over the
public internet, especially in an era where it is becoming quite clearly
evident that advanced persistent security threats are even more widespread
and more indiscriminate in their targeting than had previously been
understood. While they represented reasonable security infrastructure in
their time, the state of the art has moved on, and we need to investigate
mechanisms for effectively providing more up to date network security
infrastructure for users that, for whatever reason, are not currently in
a position to migrate to Python 3.

While the use of the system OpenSSL installation addresses many of these
concerns on Linux platforms, it doesn't address all of them, and in the
case of the binary installers for Windows and Mac OS X that are published
on python.org, the version of OpenSSL used is entirely within the control
of the Python core development team, and currently limited to OpenSSL
maintenance releases for the version initially shipped with the corresponding
Python feature release.

With increased popularity comes increased responsibility, and this policy
aims to acknowledge the fact that Python's popularity and adoption has now
reached a level where some of our design and policy decisions have
significant implications beyond the Python development community.

As one example, the Python 2 ``ssl`` module does not support the Server
Name Identification standard. While it is possible to obtain SNI support
by using the third party ``requests`` client library, actually doing so
currently requires using not only ``requests`` and its embedded dependencies,
but also half a dozen or more additional libraries. The lack of support
in the Python 2 series thus serves as an impediment to making effective
use of SNI on servers, as Python 2 clients will frequently fail to handle
it correctly.

Another more critical example is the lack of SSL hostname matching in the
Python 2 standard library - it is currently necessary to rely on a third
party library, such as ``requests`` or ``backports.ssl_match_hostname`` to
obtain that functionality in Python 2.

The Python 2 series also remains more vulnerable to remote timing attacks
on security sensitive comparisons than the Python 3 series, as it lacks a
standard library equivalent to the timing attack resistant
``hmac.compare_digest()`` function. While appropriate secure comparison
functions can be implemented in third party extensions, may users don't
even consider the problem and use ordinary equality comparisons instead
- while a standard library solution doesn't automatically fix that problem,
it *does* make the barrier to resolution much lower once the problem is
pointed out.

My position on the ongoing transition from Python 2 to Python 3 has long
been that Python 2 remains a supported platform for the core development
team, and that commercial support will remain available well after upstream
maintenance ends. However, in the absence of this network security
enhancement policy, that position is difficult to justify when it comes to
software that operates over the public internet. Just as many developers
consider it too difficult to develop truly secure modern networked software
in C/C++ (largely due to the challenges associated with manual
memory management), I anticipate that in the not too distant future, it
will be considered too difficult to develop truly secure modern networked
software using the Python 2 series (some developers would argue that we
have already reached that point).


Alternative: advise developers of networked software to migrate to Python 3
---------------------------------------------------------------------------

This alternative represents the status quo. Unfortunately, it has proven
to be unworkable in practice, as the backwards compatibility implications
mean that this is a non-trivial migration process for large applications
and integration projects.

Now that we're fully aware of the impact the limitations in Python 2 may be
having on the evolution of internet security standards, I no longer believe
that it is reasonable to expect platform and application developers to
resolve all of the latent defects in an application's Unicode correctness
solely in order to gain access to the network security enhancements
available in Python 3.

While (as far as I am aware) Ubuntu has successfully switched to Python 3.4
as its main Python interpreter for its 14.04 LTS release, Fedora still
has a lot of work to do to migrate, and it will take a non-trivial amount
of time to migrate the relevant infrastructure components. While Red Hat
are also actively working to make it easier for users to use more recent
versions of Python on our stable platforms, it's going to take time for
those efforts to start having an impact on end users' choice of version,
and those changes won't affect the core tools regardless.

The OpenStack migration to Python 3 is also still in its infancy, and even
though that's a project with an extensive and relatively robust automated
test suite, it's large enough that it is going to take quite some time
to migrate.

And that's just three of the highest profile open source projects that
make heavy use of Python. Given the likely existence of large amounts of
legacy code that lacks the kind of automated regression test suite needed
to help support a migration from Python 2 to Python 3. The key point of
this PEP is that those situations affect more people than just the
developers and users of the affected application: their existence becomes
something that developers of secure networked services need to take into
account as part of their security design.

As Terry Reedy noted, if we try to persist with the status quo, the likely
outcome is that commercial redistributors will attempt to do something
like this on behalf of their customers *anyway*, but in a potentially
inconsistent and ad hoc manner. By drawing the scope definition process
into the upstream project we are in a better position to influence the
approach taken to address the situation and to help ensure some consistency
across redistributors.

The problem is real, so *something* needs to change, and this PEP describes
my currently preferred approach to addressing the situation.


Alternative: create and release Python 2.8
------------------------------------------

With sufficient corporate support, it likely *would* be possible to create
and release Python 2.8 (it's highly unlikely such a project would garner
enough interest to be achievable with only volunteers). However, this
wouldn't actually solve the problem, as the aim is to provide a *relatively
low impact* way to incorporate enhanced security features into integrated
products and deployments that make use of Python 2. Upgrading to a new
Python feature release would mean both more work for the core development
team, as well as a more disruptive update that most potential end users
would likely just skip entirely.

Attempting to create a Python 2.8 release would also bring in suggestions
to backport many additional features from Python 3 (such as ``tracemalloc``
and the improved coroutine support).

This is not a recommended approach, as it would involve substantial
additional work for a result that is actually less effective as a solution
to the original problem (the widespread use of the aging network security
infrastructure in Python 2).


Alternative: distribute the security enhancements via PyPI
----------------------------------------------------------

While it initially appears to be an attractive and easier to manage
approach, there are actually several significant problems with this
idea.

Firstly, this PEP encompasses a non-trivial portion of the standard library.
It's not just the underlying SSL support, but also the libraries for other
network protocols like HTTP, FTP, IMAP, and POP3 that integrate with the
SSL infrastructure to provide secure links, and that's just the protocols
in the standard library.  Even if an API compatible ``ssl2`` module was
made available, it would need to be imported and injected
into ``sys.modules`` as ``ssl`` before importing any other module that
needed it.

Secondly, this is complex, low level, cross-platform code that integrates
with the underlying operating system across a variety of POSIX platforms
(including Mac OS X) and Windows. The CPython BuildBot fleet is already set
up to handle continuous integration in that context, but most of the
freely available continuous integration services just offer Linux, and
perhaps paid access to Windows. Those services work reasonably well for
software that largely runs on the abstraction layers offered by Python and
other dynamic languages, but won't suffice for the kind of code involved
here.

The OpenSSL dependency for the network security support also qualifies as
the kind of "complex binary dependency" that isn't yet handled well by the
``pip`` based software distribution ecosystem. Relying on a binary
dependency also creates potential compatibility problems for ``pip`` when
running on other interpreters like ``PyPy``.

Another practical problem with the idea is the fact that ``pip`` itself
relies on the ``ssl`` support in the standard library (with some additional
support from a bundled copy of ``requests``, which in turn bundles
``backport.ssl_match_hostname``), and hence would require any replacement
module to also be bundled within ``pip``. This wouldn't pose any
insurmountable difficulties (it's just another dependency to vendor), but
it *would* mean yet another copy of OpenSSL to keep up to
date.

This approach also has the same flaw as all other "improve security by
renaming things" approaches: they completely miss the users who most need
help, and raise significant barriers against being able to encourage users
to do the right thing when their infrastructure supports it (since
"use this other module" is a much higher impact change than "turn on this
higher security setting"). Deprecating the aging SSL infrastructure in the
standard library in favour of an external module would be even more user
hostile than taking the risk of trying to upgrade it in place.

Last, but certainly not least, this approach suffers from the same problem
as the idea of doing a Python 2.8 release: likely not solving the actual
problem. Commercial redistributors of Python are set up to redistribute
*Python*, and a pre-existing set of additional packages. Getting new
packages added to the pre-existing set *can* be done, but means going
around to each and every redistributor and asking them to update their
repackaging process accordingly. By contrast, the approach described in
this PEP would require redistributors to *opt out* of the security
enhancements, which most of them are unlikely to do.


Open Questions
==============

* What are the risks associated with allowing OpenSSL to be updated to
  new feature versions in the Windows and Mac OS X binary installers for
  maintenance releases? Currently we just upgrade to the appropriate
  OpenSSL maintenance releases, rather than switching to the latest
  feature release. In particular, is it possible Windows C extensions may
  be linking against the Python provided OpenSSL module?

* Are there any other security relevant modules that should be covered
  by either a blanket or conditional exemption?


Disclosure of Interest
======================

The author of this PEP currently works for Red Hat on test automation tools.
If this proposal is accepted, I will be strongly encouraging Red Hat to take
advantage of the resulting opportunity to help improve the overall security
of the Python ecosystem. However, I do not speak for Red Hat in this matter,
and cannot make any commitments on Red Hat's behalf.


Acknowledgements
================

Thanks to Christian Heimes for his recent efforts on greatly improving
Python's SSL support in the Python 3 series, and a variety of members of
the Python community for helping me to better understand the implications
of the default settings we provide in our SSL modules, and the impact that
tolerating the use of SSL infrastructure that was defined in 2010
(Python 2.7) or even 2008 (Python 2.6) potentially has for the security
of the web as a whole.

Christian and Donald Stufft also provided valuable feedback on a preliminary
draft of this proposal.

Thanks also to participants in the python-dev mailing list thread [1]_


References
==========

.. [1] https://mail.python.org/pipermail/python-dev/2014-March/133334.html


Copyright
=========

This document has been placed in the public domain.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia