[Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements
ncoghlan at gmail.com
Tue Mar 25 14:09:45 CET 2014
Hmm, should probably have more entries in the post history... Anyway:
* As Ben Darnell noted, the legacy SSL branch thing was a
fundamentally bad idea. We can't stop downstream redistributors doing
that if they really want to, but we don't have to encourage them
* I also explained why I think selective backports are a bad idea
* I've given a lot more context on how Red Hat actually manages to
handle such long term support cycles, and that what I'm proposing here
really isn't that radical from that perspective (and is, in fact,
about the only available solution that can be effectively slotted into
* I've been a lot more explicit about the fact that long term
maintenance releases are genuinely *different* from CPython's normal
* more details on the specific support I think this proposal would
need to become a practical reality
* removed some unhelpful side comments and obliquely acknowledged the
fact that it mat conceivably have been difficult for outsiders to see
that it might actually be possible to make this case effectively :)
Title: Network Security Enhancement Exception for Python 2.7
Author: Nick Coghlan <ncoghlan at gmail.com>,
Most CPython tracker issues are classified as errors in behaviour or
proposed enhancements. Most patches to fix behavioural errors are
applied to all active maintenance branches. Enhancement patches are
restricted to the default branch that becomes the next Python version.
This cadence works reasonably well during Python's normal 18-24 month
feature release cycle, which is still applicable to the Python 3 series.
However, the age of the standard library in Python 2 has now reached a point
where it is sufficiently far behind the state of the art in network security
protocols for it to be causing real problems in commercial use cases
where upgrading to Python 3 in the near term may not be practical.
In recognition of the additional practical considerations that have arisen
during the 4+ year maintenance cycle for Python 2.7, this PEP allows
Python 2.7 standard library components that have implications for the
overall security of the internet to be updated in line with the
corresponding Python 3 feature releases.
Specifically, the exception will apply to:
* the ``ssl`` module
* the ``hashlib`` module
* the ``hmac`` module
* the components of the ``random`` and ``os`` modules that the above
modules rely on for cryptographic randomness
* the version of OpenSSL bundled with the binary installers for Windows
and Mac OS X
Changes to these modules will still need to undergo normal backwards
compatibility assessments to ensure their default behaviour remains
consistent with earlier Python 2.7 releases, but otherwise they will be
kept entirely aligned with the latest feature release of their Python 3
counterparts. This is designed to make it easier to implement secure
networked software in Python, even for software that currently needs to
remain compatible with the Python 2 series (which includes a lot of
network infrastructure software).
While this PEP does not make any changes to the core development team's
handling of security-fix-only branches that are no longer in active
maintenance, it *does* recommend that commercial redistributors providing
extended support periods for the Python standard library either adopt a
similar approach to ensuring that the secure networking infrastructure
keeps pace with the evolution of the internet, or else explicitly
disclaim support for the use of older versions in roles that involve
connecting directly to the public internet.
Under this policy, the following network security related modules are
granted a blanket exemption to the normal restriction against adding new
features in Python 2.7 maintenance releases, for the purpose of keeping
their APIs aligned with their counterparts in the latest feature release
of Python 3:
* the ``ssl`` module
* the ``hashlib`` module
* the ``hmac`` module
Under this exemption, these modules are updated to provide identical
functionality to their Python 3 counterparts after every new Python 3
feature release. The default behaviour of the backported modules will be
adjusted as appropriate to meet the backwards compatibility requirements
of a Python 2.7 maintenance release.
As part of this policy, permission is also granted to upgrade to newer
feature releases of OpenSSL when preparing the binary installers
for new maintenance releases of Python 2.7.
Note that the ``sha`` and ``md5`` modules are not covered by this policy,
but have been deprecated in favour of ``hashlib`` since Python 2.5 and have
been removed entirely in the Python 3 series.
In addition to the above blanket exemption, a conditional exemption is
granted for these modules that may include some network security related
* the ``os`` module (specifically ``os.urandom``)
* the ``random`` module
This more limited exemption for these modules requires that the *specific*
enhancement being proposed for backporting needs to be justified as being
network security related. This is generally restricted to cases where the
feature in question is needed by an update to one of the modules covered
by the blanket exemption.
Backwards Compatibility Considerations
This PEP does *not* grant Python 2.7 any general exemptions to the usual
backwards compatibility policy for maintenance releases. Instead, by
explicitly encouraging the use of feature based checks and explicitly
opting in to less secure configurations, it is designed to make it easier
to write more secure cross-version compatible Python software, while still
limiting the risk of breaking currently working software when upgrading to
a new Python 2.7 maintenance release.
In *all* cases where this policy is applied to backport enhancements to
Python 2.7 maintenance releases, it MUST be possible to write cross-version
compatible code that operates by "feature detection" (for example, checking
for particular attributes in the module), without needing to explicitly check
the Python version.
It is then up to library and framework code to provide an appropriate warning
and fallback behaviour if a desired feature is found to be missing. While
some especially security sensitive software MAY fail outright if a desired
security feature is unavailable, most software SHOULD instead emit a warning
and continue operating using a slightly degraded security configuration.
Affected APIs SHOULD be designed to allow library and application code to
perform the following actions after detecting the presence of a relevant
network security related feature:
* explicitly opt in to more secure settings (to allow the use of enhanced
security features in older maintenance releases of Python)
* explicitly opt in to less secure settings (to allow the use of newer Python
feature releases in lower security environments)
* determine the default setting for the feature (this MAY require explicit
Python version checks to determine the Python feature release, but MUST
NOT require checking for a specific maintenance release)
Security related changes to other modules (such as higher level networking
libraries and data format processing libraries) will continue to be made
available as backports and new modules on the Python Package Index, as
independent distribution remains the preferred approach to handling
software that must continue to evolve to handle changing development
requirements independently of the Python 2 standard library. Refer to
the `Motivation and Rationale`_ section for a review of the characteristics
that make the secure networking infrastructure worthy of special
Under this policy, OpenSSL may be upgraded to more recent feature releases
in Python 2.7 maintenance releases. On Linux and most other POSIX systems,
the specific version of OpenSSL used already varies, as CPython dynamically
links to the system provided OpenSSL library by default.
For the Windows binary installers, the ``_ssl`` and ``_hashlib`` modules are
statically linked with OpenSSL and the associated symbols are not exported.
Marc-Andre Lemburg indicates that updating to newer OpenSSL releases in the
``egenix-pyopenssl`` binaries has not resulted in any reported compatibility
The Mac OS X binary installers historically followed the same policy as
other POSIX installations and dynamically linked to the Apple provided
OpenSSL libraries. However, Apple has now ceased updating these
cross-platform libraries, instead requiring that even cross-platform
developers adopt Mac OS X specific interfaces to access up to date security
infrastructure on their platform. Accordingly, and independently of this
PEP, the Mac OS X binary installers were already going to be switched to
statically linker newer versions of OpenSSL _
This policy does NOT represent a commitment by volunteer contributors to
actually backport network security related changes from the Python 3 series
to the Python 2 series. Rather, it is intended to send a clear signal to
potential corporate contributors that the core development team are willing
to accept offers of corporate assistance in putting this policy into
effect and handling the resulting increase in the Python 2 maintenance
Backporting security related fixes and enhancements to earlier versions is
a common service for commercial redistributors to offer to their customers.
This policy represents an explicit invitation to contribute some of those
changes back to the upstream Python community in cases where they are likely
to have a broad impact that helps improve the security of the internet as a
whole, with the assurance that the existing core development team not only
won't object to such contributions, but will actively encourage their
incorporation into the next Python 2.7 maintenance release.
All modules covered by this policy MUST include a "Security Considerations"
section in their documentation in order to take advantage of this policy.
In addition to any other module specific contents, this section MUST
enumerate key security enhancements and fixes (with CVE identifiers where
applicable), along with the feature and maintenance releases that first
This PEP does not propose any changes to the handling of security
releases - those will continue to be source only releases that
include only critical security fixes.
However, the recommendations for library and application developers are
deliberately designed to accommodate commercial redistributors that choose
to apply this policy to additional Python release series that are either in
security fix only mode, or have been declared "end of life" by the core
Whether or not redistributors choose to exercise that option will be up
to the individual redistributor.
Third party integration testing services should offer users the ability
to test against specific Python 2.7 maintenance releases, to ensure that
libraries, frameworks and applications can still test their handling of the
legacy security infrastructure correctly (either failing or degrading
gracefully, depending on the security sensitivity of the software), even
after the latest Python 2.7 maintenance release has been synchronised with
a new Python 3 feature release for the modules covered by this policy.
Handling lower security environments with low risk tolerance
For better or for worse (mostly worse), there are some environments where
the risk of latent security defects is more tolerated than even a slightly
increased risk of regressions in maintenance releases. This policy largely
excludes these environments from consideration where the modules covered by
the exemption are concerned - this approach is entirely inappropriate for
software connected to the public internet, and defence in depth security
principles suggest that it is not appropriate for most private networks
Downstream redistributors may still choose to cater to such environments,
but they will need to handle the process of downgrading the security
related modules and doing the associated regression testing themselves.
The main CPython continuous integration infrastructure will not cover this
Evolution of this Policy
The key requirement for a module to be considered for inclusion in this
policy (whether under a blanket or conditional exemption) is that it must
have security implications *beyond* the specific application that is written
in Python and the system that application is running on. Thus the focus on
network security protocols and related cryptographic infrastructure - Python
is a popular choice for the development of web services and clients, and
thus the capabilities of widely used Python versions have implications for
the security design of other services that may themselves be using newer
versions of Python or other development languages, but need to interoperate
with clients or servers written using older versions of Python.
The intent behind this requirement is to minimise any impact that the
introduction of this policy may have on the stability and compatibility of
maintenance releases. It would be thoroughly counterproductive if end
users became as cautious about updating to new Python 2.7 maintenance
releases as they are about updating to new feature releases within the
same release series.
Motivation and Rationale
This PEP can be seen as a more targeted version of the "faster standard
library release cycle" proposals discussed in PEP 407 and PEP 413,
focusing specifically on those areas which have implications beyond the
The creation of this PEP was prompted primarily by the aging SSL support in
the Python 2 series. As of March 2014, the Python 2.7 SSL module is
approaching four years of age, and the SSL support in the still popular
Python 2.6 release had its feature set locked six years ago.
These are simply too old to provide a foundation that can be recommended
in good conscience for secure networking software that operates over the
public internet, especially in an era where it is becoming quite clearly
evident that advanced persistent security threats are even more widespread
and more indiscriminate in their targeting than had previously been
understood. While they represented reasonable security infrastructure in
their time, the state of the art has moved on, and we need to investigate
mechanisms for effectively providing more up to date network security
infrastructure for users that, for whatever reason, are not currently in
a position to migrate to Python 3.
While the use of the system OpenSSL installation addresses many of these
concerns on Linux platforms, it doesn't address all of them (in particular,
it is still difficult for sotware to explicitly require some higher level
security settings). In the case of the binary installers for Windows and
Mac OS X that are published on python.org, the version of OpenSSL used is
entirely within the control of the Python core development team, but is
currently limited to OpenSSL maintenance releases for the version initially
shipped with the corresponding Python feature release.
With increased popularity comes increased responsibility, and this policy
aims to acknowledge the fact that Python's popularity and adoption has now
reached a level where some of our design and policy decisions have
significant implications beyond the Python development community.
As one example, the Python 2 ``ssl`` module does not support the Server
Name Identification standard. While it is possible to obtain SNI support
by using the third party ``requests`` client library, actually doing so
currently requires using not only ``requests`` and its embedded dependencies,
but also half a dozen or more additional libraries. The lack of support
in the Python 2 series thus serves as an impediment to making effective
use of SNI on servers, as Python 2 clients will frequently fail to handle
Another more critical example is the lack of SSL hostname matching in the
Python 2 standard library - it is currently necessary to rely on a third
party library, such as ``requests`` or ``backports.ssl_match_hostname`` to
obtain that functionality in Python 2.
The Python 2 series also remains more vulnerable to remote timing attacks
on security sensitive comparisons than the Python 3 series, as it lacks a
standard library equivalent to the timing attack resistant
``hmac.compare_digest()`` function. While appropriate secure comparison
functions can be implemented in third party extensions, many users don't
even consider the issue and use ordinary equality comparisons instead
- while a standard library solution doesn't automatically fix that problem,
it *does* make the barrier to resolution much lower once the problem is
My position on the ongoing transition from Python 2 to Python 3 has long
been that Python 2 remains a supported platform for the core development
team, and that commercial support will remain available well after upstream
maintenance ends. However, in the absence of this network security
enhancement policy, that position is difficult to justify when it comes to
software that operates over the public internet. Just as many developers
consider it too difficult to develop truly secure modern networked software
in C/C++ (largely due to the challenges associated with manual
memory management), I anticipate that in the not too distant future, it
will be considered too difficult to develop truly secure modern networked
software using the Python 2 series (some developers would argue that we
have already reached that point).
Python 2.7 represents the only long term maintenance release the core
development team has provided, and it is natural that there will be things
that worked over a historically shorter maintenance lifespan that don't work
over this longer support period. In the specific case of the problem
described in this PEP, the simplest available solution is to acknowledge
that long term maintenance of network security related modules *requires*
the ability to add new features, even while retaining backwards compatibility
for existing interfaces.
It is worth comparing the approach described in this PEP with Red Hat's
handling of its long term support commitments: it isn't the RHEL 6.0 release
itself that receives 10 years worth of support, but the overall RHEL 6
*series*. The individual RHEL 6.x point releases within the series then
receive a wide variety of new features, including security enhancements,
all while meeting strict backwards compatibility guarantees for existing
software. Subscribers *are* able to continue using a given point release
after the next point release has become available, but a corresponding
add-on subscription for `Extended Update Support
is needed to cover the additional backporting work involved.
To date, downstream redistributors have respected our upstream policy of
"no new features in Python maintenance releases". This PEP explicitly
accepts that a more nuanced policy is appropriate in the case of network
security related features, and the specific one it describes is deliberately
designed such that it at least has some chance of being applied to Red Hat
Enterprise Linux and its downstream derivatives.
Alternative: advise developers of networked software to migrate to Python 3
This alternative represents the status quo. Unfortunately, it has proven
to be unworkable in practice, as the backwards compatibility implications
mean that this is a non-trivial migration process for large applications
and integration projects. While the tools for migration have evolved to
a point where it is possible to migrate even large applications
opportunistically and incrementally (rather than all at once) by updating
code to run in the large common subset of Python 2 and Python 3, using the
most recent technology often isn't a priority in commercial environments.
Previously, this was considered an acceptable harm, as while it was an
unfortunate problem for the affected developers to have to face, it was
seen as an issue between them and their management chain to make the case
for infrastructure modernisation, and this case would become naturally
more compelling as the Python 3 series evolved.
However, now that we're fully aware of the impact the limitations of the
Python 2 standard library may be having on the evolution of internet
security standards, I no longer believe that it is reasonable to expect
platform and application developers to resolve all of the latent defects
in an application's Unicode correctness solely in order to gain access to
the network security enhancements already available in Python 3.
While Ubuntu (and to some extent Debian as well) are committed to porting all
default system services and scripts to Python 3, and to removing Python 2
from its default distribution images (but not from its archives), this is
a mammoth task and won't be completed for the Ubuntu 14.04 LTS release
(at least for the desktop image - it may be achieved for the mobile and
Fedora has even more work to do to migrate, and it will take a non-trivial
amount of time to migrate the relevant infrastructure components. While
Red Hat are also actively working to make it easier for users to use more
recent versions of Python on our stable platforms, it's going to take time
for those efforts to start having an impact on end users' choice of version,
and any such changes also don't benefit the core platform infrastructure
that runs in the integrated system Python by necessity.
The OpenStack migration to Python 3 is also still in its infancy, and even
though that's a project with an extensive and relatively robust automated
test suite, it's still large enough that it is going to take quite some time
And that's just three of the highest profile open source projects that
make heavy use of Python. Given the likely existence of large amounts of
legacy code that lacks the kind of automated regression test suite needed
to help support a migration from Python 2 to Python 3, there are likely to
be many cases where reimplementation (perhaps even in Python 3) proves
easier than migration. The key point of this PEP is that those situations
affect more people than just the developers and users of the affected
application: the existence of clients and servers with outdated network
security infrastructure becomes something that developers of secure
networked services need to take into account as part of their security
design, and that's a problem that inhibits the adoption of better security
As Terry Reedy noted, if we try to persist with the status quo, the likely
outcome is that commercial redistributors will attempt to do something
like this on behalf of their customers *anyway*, but in a potentially
inconsistent and ad hoc manner. By drawing the scope definition process
into the upstream project we are in a better position to influence the
approach taken to address the situation and to help ensure some consistency
The problem is real, so *something* needs to change, and this PEP describes
my preferred approach to addressing the situation.
Alternative: create and release Python 2.8
With sufficient corporate support, it likely *would* be possible to create
and release Python 2.8 (it's highly unlikely such a project would garner
enough interest to be achievable with only volunteers). However, this
wouldn't actually solve the problem, as the aim is to provide a *relatively
low impact* way to incorporate enhanced security features into integrated
products and deployments that make use of Python 2.
Upgrading to a new Python feature release would mean both more work for the
core development team, as well as a more disruptive update that most
potential end users would likely just skip entirely.
Attempting to create a Python 2.8 release would also bring in suggestions
to backport many additional features from Python 3 (such as ``tracemalloc``
and the improved coroutine support), making the migration from Python 2.7
to this hypothetical 2.8 release even riskier and more disruptive.
This is not a recommended approach, as it would involve substantial
additional work for a result that is actually less effective in achieving
the original aim (which is to eliminate the current widespread use of the
aging network security infrastructure in the Python 2 series).
Furthermore, while I can't make any commitments to actually addressing
this issue on Red Hat platforms, I *can* categorically rule out the idea
of a Python 2.8 being of any use to me in even attempting to get it
Alternative: distribute the security enhancements via PyPI
While this initially appears to be an attractive and easier to manage
approach, it actually suffers from several significant problems.
Firstly, this is complex, low level, cross-platform code that integrates
with the underlying operating system across a variety of POSIX platforms
(including Mac OS X) and Windows. The CPython BuildBot fleet is already set
up to handle continuous integration in that context, but most of the
freely available continuous integration services just offer Linux, and
perhaps paid access to Windows. Those services work reasonably well for
software that largely runs on the abstraction layers offered by Python and
other dynamic languages, as well as the more comprehensive abstraction
offered by the JVM, but won't suffice for the kind of code involved here.
The OpenSSL dependency for the network security support also qualifies as
the kind of "complex binary dependency" that isn't yet handled well by the
``pip`` based software distribution ecosystem. Relying on a third party
binary dependency also creates potential compatibility problems for ``pip``
when running on other interpreters like ``PyPy``.
Another practical problem with the idea is the fact that ``pip`` itself
relies on the ``ssl`` support in the standard library (with some additional
support from a bundled copy of ``requests``, which in turn bundles
``backport.ssl_match_hostname``), and hence would require any replacement
module to also be bundled within ``pip``. This wouldn't pose any
insurmountable difficulties (it's just another dependency to vendor), but
it *would* mean yet another copy of OpenSSL to keep up to date.
This approach also has the same flaw as all other "improve security by
renaming things" approaches: they completely miss the users who most need
help, and raise significant barriers against being able to encourage users
to do the right thing when their infrastructure supports it (since
"use this other module" is a much higher impact change than "turn on this
higher security setting"). Deprecating the aging SSL infrastructure in the
standard library in favour of an external module would be even more user
hostile than accepting the slightly increased risk of regressions associated
with upgrading it in place.
Last, but certainly not least, this approach suffers from the same problem
as the idea of doing a Python 2.8 release: likely not solving the actual
problem. Commercial redistributors of Python are set up to redistribute
*Python*, and a pre-existing set of additional packages. Getting new
packages added to the pre-existing set *can* be done, but means approaching
each and every redistributor and asking them to update their
repackaging process accordingly. By contrast, the approach described in
this PEP would require redistributors to deliberately *opt out* of the
security enhancements by deliberately downgrading the provided network
security infrastructure, which most of them are unlikely to do.
Alternative: provide a "legacy SSL infrastructure" branch
Earlier versions of this PEP included the concept of a ``2.7-legacy-ssl``
branch that preserved the exact feature set of the Python 2.7.6 network
It is the opinion of the PEP author that anyone that actually wants this is
almost certainly making a mistake, and if they insist they really do want
it in their specific situation, they're welcome to either make it themselves
or arrange for a downstream redistributor to make it for them.
If they are made publicly available, any such rebuilds should be referred to
as "Python 2.7 with Legacy SSL" to clearly distinguish them from the official
Python 2.7 releases that include more up to date network security
After the first Python 2.7 maintenance release that has the security
infrastructure updated to match Python 3.4, it would also be appropriate to
refer to Python 2.7.6 and earlier releases as "Python 2.7 with Legacy SSL".
Alternative: selectively backport particular APIs
An instinctively minimalist reaction to this proposal is to only backport
particular APIs in the affected modules that are judged to be "security
critical". However, this ends up providing a worse end user experience,
as well as a worse developer experience.
For end users, the selective backporting approach means learning not only
the legacy Python 2.7 API and the current Python 3 APIs, but also the
hybrid API created by the selective backporting process. It is much
simpler to just learn the APIs of the original Python 2.7 feature release
and the relevant Python 3 features releases, without trying to define a new
hybrid API that only exists in later Python 2.7 maintenance branches.
For developers, the main benefit of the "just align the available
functionality with Python 3" approach is that it reduces the amount of
design work needing when updating Python 2.7 with new security features.
The "feature or fix?" and "security related or not?" debates are both
handled in advance by this policy, leaving only the matter of ensuring
that backwards compatibility is preserved as needed by adjusting the
default behaviour in the Python 2.7 backport when appropriate.
* MvL has indicated he is not prepared to tackle the task of trying to
integrate a newer OpenSSL into the also aging Python 2.7 build
infrastructure on Windows (unfortunately, we've looked into upgrading
that build infrastructure, and the backwards compatibility issues
appear to be effectively insurmountable). We would require a commitment
from another trusted contributor to handle at least this task, and
potentially also taking over the task of creating the official
Python 2.7 Windows installers for the remaining Python 2.7 maintenance
* We would need commitments to create and review full backports of the
components covered by this policy from Python 3.4 to Python 2.7, as well
as support for handling any more specific security issues affecting these
* I believe I've addressed all the technical and scope questions I had, or
others raised. That just leaves the question of "Do we agree this plan
actually makes sense, given the constraints on downstream redistributors
that would also like to see this problem solved?" :)
Disclosure of Interest
The author of this PEP currently works for Red Hat on test automation tools.
If this proposal is accepted, I will be strongly encouraging Red Hat to take
advantage of the resulting opportunity to help improve the overall security
of the Python ecosystem. However, I do not speak for Red Hat in this matter,
and cannot make any commitments on Red Hat's behalf.
Thanks to Christian Heimes for his recent efforts on greatly improving
Python's SSL support in the Python 3 series, and a variety of members of
the Python community for helping me to better understand the implications
of the default settings we provide in our SSL modules, and the impact that
tolerating the use of SSL infrastructure that was defined in 2010
(Python 2.7) or even 2008 (Python 2.6) potentially has for the security
of the web as a whole.
Christian and Donald Stufft also provided valuable feedback on a preliminary
draft of this proposal.
Thanks also to participants in the python-dev mailing list threads [1,2,5]_
..  https://mail.python.org/pipermail/python-dev/2014-March/133334.html
..  https://mail.python.org/pipermail/python-dev/2014-March/133389.html
..  https://mail.python.org/pipermail/python-dev/2014-March/133438.html
..  https://mail.python.org/pipermail/python-dev/2014-March/133347.html
..  https://mail.python.org/pipermail/python-dev/2014-March/133442.html
Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia
More information about the Python-Dev