[Python-checkins] peps: PEP 466: further simplify the proposal

nick.coghlan python-checkins at python.org
Tue Mar 25 13:42:26 CET 2014 

http://hg.python.org/peps/rev/4eb8de143801
changeset:   5428:4eb8de143801
user:        Nick Coghlan <ncoghlan at gmail.com>
date:        Tue Mar 25 22:40:26 2014 +1000
summary:
  PEP 466: further simplify the proposal

- dropped the "legacy ssl" branch idea again
- explained why selective backports are a bad idea
- better explained how long term support differs from short term
- better explained some downstream constraints
- explained that downstream redistributors have been respecting
  our upstream policies, potentially against their own better
  judgement
- used Red Hat's support cycle as a reference model
- removed the snarky comments that hit unintended targets
- note the need for Windows specific contributions as an open question
- not the general need for contributions of time as an open question
- last open question is whether or not we're happy this is the best
  available solution given downstream constraints

files:
  pep-0466.txt |  267 ++++++++++++++++++++++++++------------
  1 files changed, 179 insertions(+), 88 deletions(-)


diff --git a/pep-0466.txt b/pep-0466.txt
--- a/pep-0466.txt
+++ b/pep-0466.txt
@@ -25,10 +25,13 @@
 protocols for it to be causing real problems in commercial use cases
 where upgrading to Python 3 in the near term may not be practical.
 
-Accordingly, this PEP relaxes the normal restrictions by allowing
-enhancements to be applied in Python 2.7 maintenance releases for standard
-library components that have implications for the overall security of the
-internet. In particular, the exception will apply to:
+In recognition of the additional practical considerations that have arisen
+during the 4+ year maintenance cycle for Python 2.7, this PEP allows
+Python 2.7 standard library components that have implications for the
+overall security of the internet to be updated in line with the
+corresponding Python 3 feature releases.
+
+Specifically, the exception will apply to:
 
 * the ``ssl`` module
 * the ``hashlib`` module
@@ -39,52 +42,46 @@
   and Mac OS X
 
 Changes to these modules will still need to undergo normal backwards
-compatibility assessments, but otherwise they will be kept entirely aligned
-with the latest feature release of their Python 3 counterparts. This is
-designed to make it easier to implement secure networked software in
-Python, even for software that currently needs to remain compatible with
-the Python 2 series (which includes a lot of network infrastructure
-software).
-
-The development branches will be arranged in such a way that even though
-any further Python 2.7 releases published by the core development team
-provide updated network security infrastructure, it will remain possible
-for downstream redistributors to publish "Python 2.7 with legacy SSL
-infrastructure" if they choose to do so.
+compatibility assessments to ensure their default behaviour remains
+consistent with earlier Python 2.7 releases, but otherwise they will be
+kept entirely aligned with the latest feature release of their Python 3
+counterparts. This is designed to make it easier to implement secure
+networked software in Python, even for software that currently needs to
+remain compatible with the Python 2 series (which includes a lot of
+network infrastructure software).
 
 While this PEP does not make any changes to the core development team's
 handling of security-fix-only branches that are no longer in active
 maintenance, it *does* recommend that commercial redistributors providing
 extended support periods for the Python standard library either adopt a
 similar approach to ensuring that the secure networking infrastructure
-keeps pace with the evolution of the internet, or else disclaim support
-for the use of older versions in roles that involving connecting
-directly to the public internet.
+keeps pace with the evolution of the internet, or else explicitly
+disclaim support for the use of older versions in roles that involve
+connecting directly to the public internet.
 
 
 Exemption Policy
 ================
 
 Under this policy, the following network security related modules are
-granted a blanket exemption to the restriction against adding new features
-in maintenance releases, for the purpose of keeping their APIs aligned with
-their counterparts in the latest feature release of Python 3:
+granted a blanket exemption to the normal restriction against adding new
+features in Python 2.7 maintenance releases, for the purpose of keeping
+their APIs aligned with their counterparts in the latest feature release
+of Python 3:
 
 * the ``ssl`` module
 * the ``hashlib`` module
 * the ``hmac`` module
 
-This exemption applies to *all* proposals to backport backwards compatible
-changes in these modules to Python 2.7 maintenance releases. This choice is
-made deliberately to ensure that the "feature or fix?" argument isn't simply
-replaced by a "security related or not?" argument. These particular modules
-are inherently security related, and all enhancements to them improve
-Python's capabilities as a platform for development of secure networked
-software.
+Under this exemption, these modules are updated to provide identical
+functionality to their Python 3 counterparts after every new Python 3
+feature release. The default behaviour of the backported modules will be
+adjusted as appropriate to meet the backwards compatibility requirements
+of a Python 2.7 maintenance release.
 
 As part of this policy, permission is also granted to upgrade to newer
 feature releases of OpenSSL when preparing the binary installers
-for new maintenance releases of CPython.
+for new maintenance releases of Python 2.7.
 
 Note that the ``sha`` and ``md5`` modules are not covered by this policy,
 but have been deprecated in favour of ``hashlib`` since Python 2.5 and have
@@ -107,18 +104,18 @@
 Backwards Compatibility Considerations
 ======================================
 
-This PEP does *not* grant any general exemptions to the usual backwards
-compatibility policy for maintenance releases. Instead, by explicitly
-encouraging the use of feature based checks and explicitly opting in to
-less secure configurations, it is designed to make it easier to provide
-more "secure by default" behaviour in future feature releases, while still
+This PEP does *not* grant Python 2.7 any general exemptions to the usual
+backwards compatibility policy for maintenance releases. Instead, by
+explicitly encouraging the use of feature based checks and explicitly
+opting in to less secure configurations, it is designed to make it easier
+to write more secure cross-version compatible Python software, while still
 limiting the risk of breaking currently working software when upgrading to
-a new maintenance release.
+a new Python 2.7 maintenance release.
 
 In *all* cases where this policy is applied to backport enhancements to
-maintenance releases, it MUST be possible to write cross-version compatible
-code that operates by "feature detection" (for example, checking for
-particular attributes in the module), without needing to explicitly check
+Python 2.7 maintenance releases, it MUST be possible to write cross-version
+compatible code that operates by "feature detection" (for example, checking
+for particular attributes in the module), without needing to explicitly check
 the Python version.
 
 It is then up to library and framework code to provide an appropriate warning
@@ -155,7 +152,7 @@
 
 Under this policy, OpenSSL may be upgraded to more recent feature releases
 in Python 2.7 maintenance releases. On Linux and most other POSIX systems,
-the specific version of OpenSSL used already varies, as Python dynamically
+the specific version of OpenSSL used already varies, as CPython dynamically
 links to the system provided OpenSSL library by default.
 
 For the Windows binary installers, the ``_ssl`` and ``_hashlib`` modules are
@@ -191,17 +188,18 @@
 Backporting security related fixes and enhancements to earlier versions is
 a common service for commercial redistributors to offer to their customers.
 This policy represents an explicit invitation to contribute some of those
-changes back to the upstream community in cases where they are likely to
-have a broad impact that helps improve the security of the internet as a
-whole, rather than sitting back and waiting for unpaid volunteers to do it
-for them.
+changes back to the upstream Python community in cases where they are likely
+to have a broad impact that helps improve the security of the internet as a
+whole, with the assurance that the existing core development team not only
+won't object to such contributions, but will actively encourage their
+incorporation into the next Python 2.7 maintenance release.
 
 
 Documentation
 -------------
 
 All modules covered by this policy MUST include a "Security Considerations"
-section in their documentation.
+section in their documentation in order to take advantage of this policy.
 
 In addition to any other module specific contents, this section MUST
 enumerate key security enhancements and fixes (with CVE identifiers where
@@ -230,36 +228,31 @@
 -------------------
 
 Third party integration testing services should offer users the ability
-to test against both the latest Python 2.7 maintenance release and the
-latest "Python 2.7 with legacy SSL infrastructure" release, to ensure that
-libraries, frameworks and applications handle the legacy security
-infrastructure correctly (either failing or degrading gracefully, depending
-on the security sensitivity of the software).
+to test against specific Python 2.7 maintenance releases, to ensure that
+libraries, frameworks and applications can still test their handling of the
+legacy security infrastructure correctly (either failing or degrading
+gracefully, depending on the security sensitivity of the software), even
+after the latest Python 2.7 maintenance release has been synchronised with
+a new Python 3 feature release for the modules covered by this policy.
 
 
 Handling lower security environments with low risk tolerance
 ------------------------------------------------------------
 
 For better or for worse (mostly worse), there are some environments where
-the risk of latent security defects is more tolerated than the risk of
-regressions in maintenance releases. This policy largely excludes these
-environments from consideration where the modules covered by the exemption
-are concerned.
+the risk of latent security defects is more tolerated than even a slightly
+increased risk of regressions in maintenance releases. This policy largely
+excludes these environments from consideration where the modules covered by
+the exemption are concerned - this approach is entirely inappropriate for
+software connected to the public internet, and defence in depth security
+principles suggest that it is not appropriate for most private networks
+either.
 
-However, one concession is made for the benefit of such environments: while
-the main ``2.7`` branch in Mercurial will include the updated network
-security infrastructure, the development process will be updated to include
-a ``2.7-legacy-ssl`` branch with the more limited feature set of the
-original 2.7 network security infrastructure, allowing downstream
-redistributors to continue to provide Python 2.7 with the legacy SSL
-infrastructure if they choose to do so. This branch will be tested
-on the CPython continuous integration infrastructure, but no releases will
-be made from it by the core development team.
-
-As noted above, corporate redistributors and users are expected to provide
-the additional development effort needed to make this practical. It is not
-acceptable to expect volunteer contributors to resolve a problem created
-largely by conservative corporate development practices.
+Downstream redistributors may still choose to cater to such environments,
+but they will need to handle the process of downgrading the security
+related modules and doing the associated regression testing themselves.
+The main CPython continuous integration infrastructure will not cover this
+scenario.
 
 
 Evolution of this Policy
@@ -279,8 +272,9 @@
 The intent behind this requirement is to minimise any impact that the
 introduction of this policy may have on the stability and compatibility of
 maintenance releases. It would be thoroughly counterproductive if end
-users became as cautious about updating to new Python maintenance releases
-as they are about updating to new feature releases.
+users became as cautious about updating to new Python 2.7 maintenance
+releases as they are about updating to new feature releases within the
+same release series.
 
 
 Motivation and Rationale
@@ -312,12 +306,13 @@
 a position to migrate to Python 3.
 
 While the use of the system OpenSSL installation addresses many of these
-concerns on Linux platforms, it doesn't address all of them, and in the
-case of the binary installers for Windows and Mac OS X that are published
-on python.org, the version of OpenSSL used is entirely within the control
-of the Python core development team, and currently limited to OpenSSL
-maintenance releases for the version initially shipped with the corresponding
-Python feature release.
+concerns on Linux platforms, it doesn't address all of them (in particular,
+it is still difficult for sotware to explicitly require some higher level
+security settings). In the case of the binary installers for Windows and
+Mac OS X that are published on python.org, the version of OpenSSL used is
+entirely within the control of the Python core development team, but is
+currently limited to OpenSSL maintenance releases for the version initially
+shipped with the corresponding Python feature release.
 
 With increased popularity comes increased responsibility, and this policy
 aims to acknowledge the fact that Python's popularity and adoption has now
@@ -342,7 +337,7 @@
 on security sensitive comparisons than the Python 3 series, as it lacks a
 standard library equivalent to the timing attack resistant
 ``hmac.compare_digest()`` function. While appropriate secure comparison
-functions can be implemented in third party extensions, may users don't
+functions can be implemented in third party extensions, many users don't
 even consider the issue and use ordinary equality comparisons instead
 - while a standard library solution doesn't automatically fix that problem,
 it *does* make the barrier to resolution much lower once the problem is
@@ -361,6 +356,34 @@
 software using the Python 2 series (some developers would argue that we
 have already reached that point).
 
+Python 2.7 represents the only long term maintenance release the core
+development team has provided, and it is natural that there will be things
+that worked over a historically shorter maintenance lifespan that don't work
+over this longer support period. In the specific case of the problem
+described in this PEP, the simplest available solution is to acknowledge
+that long term maintenance of network security related modules *requires*
+the ability to add new features, even while retaining backwards compatibility
+for existing interfaces.
+
+It is worth comparing the approach described in this PEP with Red Hat's
+handling of its long term support commitments: it isn't the RHEL 6.0 release
+itself that receives 10 years worth of support, but the overall RHEL 6
+*series*. The individual RHEL 6.x point releases within the series then
+receive a wide variety of new features, including security enhancements,
+all while meeting strict backwards compatibility guarantees for existing
+software. Subscribers *are* able to continue using a given point release
+after the next point release has become available, but a corresponding
+add-on subscription for `Extended Update Support
+<https://access.redhat.com/site/support/policy/updates/errata/#Extended_Update_Support>`__
+is needed to cover the additional backporting work involved.
+
+To date, downstream redistributors have respected our upstream policy of
+"no new features in Python maintenance releases". This PEP explicitly
+accepts that a more nuanced policy is appropriate in the case of network
+security related features, and the specific one it describes is deliberately
+designed such that it at least has some chance of being applied to Red Hat
+Enterprise Linux and its downstream derivatives.
+
 
 Alternative: advise developers of networked software to migrate to Python 3
 ---------------------------------------------------------------------------
@@ -429,7 +452,7 @@
 across redistributors.
 
 The problem is real, so *something* needs to change, and this PEP describes
-my currently preferred approach to addressing the situation.
+my preferred approach to addressing the situation.
 
 
 Alternative: create and release Python 2.8
@@ -440,10 +463,11 @@
 enough interest to be achievable with only volunteers). However, this
 wouldn't actually solve the problem, as the aim is to provide a *relatively
 low impact* way to incorporate enhanced security features into integrated
-products and deployments that make use of Python 2. Upgrading to a new
-Python feature release would mean both more work for the core development
-team, as well as a more disruptive update that most potential end users
-would likely just skip entirely.
+products and deployments that make use of Python 2.
+
+Upgrading to a new Python feature release would mean both more work for the
+core development team, as well as a more disruptive update that most
+potential end users would likely just skip entirely.
 
 Attempting to create a Python 2.8 release would also bring in suggestions
 to backport many additional features from Python 3 (such as ``tracemalloc``
@@ -455,6 +479,11 @@
 the original aim (which is to eliminate the current widespread use of the
 aging network security infrastructure in the Python 2 series).
 
+Furthermore, while I can't make any commitments to actually addressing
+this issue on Red Hat platforms, I *can* categorically rule out the idea
+of a Python 2.8 being of any use to me in even attempting to get it
+addressed.
+
 
 Alternative: distribute the security enhancements via PyPI
 ----------------------------------------------------------
@@ -504,17 +533,78 @@
 each and every redistributor and asking them to update their
 repackaging process accordingly. By contrast, the approach described in
 this PEP would require redistributors to deliberately *opt out* of the
-security enhancements (by switching to redistributing directly from the
-``2.7-legacy-ssl`` branch rather than the main ``2.7`` branch), which most
-of them are unlikely to do.
+security enhancements by deliberately downgrading the provided network
+security infrastructure, which most of them are unlikely to do.
+
+
+Alternative: provide a "legacy SSL infrastructure" branch
+---------------------------------------------------------
+
+Earlier versions of this PEP included the concept of a ``2.7-legacy-ssl``
+branch that preserved the exact feature set of the Python 2.7.6 network
+security infrastructure.
+
+It is the opinion of the PEP author that anyone that actually wants this is
+almost certainly making a mistake, and if they insist they really do want
+it in their specific situation, they're welcome to either make it themselves
+or arrange for a downstream redistributor to make it for them.
+
+If they are made publicly available, any such rebuilds should be referred to
+as "Python 2.7 with Legacy SSL" to clearly distinguish them from the official
+Python 2.7 releases that include more up to date network security
+infrastructure.
+
+After the first Python 2.7 maintenance release that has the security
+infrastructure updated to match Python 3.4, it would also be appropriate to
+refer to Python 2.7.6 and earlier releases as "Python 2.7 with Legacy SSL".
+
+
+Alternative: selectively backport particular APIs
+-------------------------------------------------
+
+An instinctively minimalist reaction to this proposal is to only backport
+particular APIs in the affected modules that are judged to be "security
+critical". However, this ends up providing a worse end user experience,
+as well as a worse developer experience.
+
+For end users, the selective backporting approach means learning not only
+the legacy Python 2.7 API and the current Python 3 APIs, but also the
+hybrid API created by the selective backporting process. It is much
+simpler to just learn the APIs of the original Python 2.7 feature release
+and the relevant Python 3 features releases, without trying to define a new
+hybrid API that only exists in later Python 2.7 maintenance branches.
+
+For developers, the main benefit of the "just align the available
+functionality with Python 3" approach is that it reduces the amount of
+design work needing when updating Python 2.7 with new security features.
+The "feature or fix?" and "security related or not?" debates are both
+handled in advance by this policy, leaving only the matter of ensuring
+that backwards compatibility is preserved as needed by adjusting the
+default behaviour in the Python 2.7 backport when appropriate.
 
 
 Open Questions
 ==============
 
+* MvL has indicated he is not prepared to tackle the task of trying to
+  integrate a newer OpenSSL into the also aging Python 2.7 build
+  infrastructure on Windows (unfortunately, we've looked into upgrading
+  that build infrastructure, and the backwards compatibility issues
+  appear to be effectively insurmountable). We would require a commitment
+  from another trusted contributor to handle at least this task, and
+  potentially also taking over the task of creating the official
+  Python 2.7 Windows installers for the remaining Python 2.7 maintenance
+  releases.
+
+* We would need commitments to create and review full backports of the
+  components covered by this policy from Python 3.4 to Python 2.7, as well
+  as support for handling any more specific security issues affecting these
+  modules.
+
 * I believe I've addressed all the technical and scope questions I had, or
-  others raised. That just leaves the question of "If we agree to this plan,
-  who is actually going to handle all the extra work involved?" :)
+  others raised. That just leaves the question of "Do we agree this plan
+  actually makes sense, given the constraints on downstream redistributors
+  that would also like to see this problem solved?" :)
 
 
 Disclosure of Interest
@@ -541,7 +631,7 @@
 Christian and Donald Stufft also provided valuable feedback on a preliminary
 draft of this proposal.
 
-Thanks also to participants in the python-dev mailing list threads [1,2]_
+Thanks also to participants in the python-dev mailing list threads [1,2,5]_
 
 
 References
@@ -551,6 +641,7 @@
 .. [2] https://mail.python.org/pipermail/python-dev/2014-March/133389.html
 .. [3] https://mail.python.org/pipermail/python-dev/2014-March/133438.html
 .. [4] https://mail.python.org/pipermail/python-dev/2014-March/133347.html
+.. [5] https://mail.python.org/pipermail/python-dev/2014-March/133442.html
 
 
 Copyright

-- 
Repository URL: http://hg.python.org/peps

More information about the Python-checkins mailing list