[Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

Guido van Rossum guido at python.org
Mon Nov 16 19:00:46 EST 2015


Hm, making Christian the BDFL-delegate would mean two out of three
authors *and* the BDFL-delegate all working for Red Hat, which clearly
has a stake (and IIUC has already committed to this approach ahead of
PEP approval). SO then it would look like this is just rubber-stamping
Red Hat's internal decision process (if it's a process -- sounds more
like an accident :-).

So, Alex, do you want to approve this PEP?

On Mon, Nov 16, 2015 at 3:54 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 17 November 2015 at 09:07, Guido van Rossum <guido at python.org> wrote:
>> So I dropped the ball on this too -- I was going to either have a look
>> or tell you quickly to find a BDFL-delegate, but ended up doing
>> neither. Skimming it now I really don't think I'm the right person to
>> review this. Maybe you could ask Alex Gaynor?
>
> If Alex is interested, that could definitely work. I've also asked
> Christian Heimes if he might be interested in handling it, as he
> worked on several of the SSL module changes that were backported, and
> he also works on FreeIPA [1] now, which will potentially be affected
> by the changes to RHEL & CentOS 7.
>
> Regards,
> Nick.
>
> [1] FreeIPA is an open source Identity Management & Authentication
> system: http://www.freeipa.org/
>
>>
>> On Tue, Nov 10, 2015 at 4:47 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>> Hi folks,
>>>
>>> I have a confession to make - I dropped the ball on the HTTPS
>>> verification backport proposals in PEP 493, and let the upstream and
>>> downstream approval processes get out of sequence.
>>>
>>> As a result, the RHEL 7.2 beta released back in September incorporates
>>> the HTTPS verification feature backport based on the current PEP 493
>>> draft, even though that hasn't formally been pronounced as an Active
>>> recommendation by python-dev yet.
>>>
>>> Accordingly, I'm belatedly submitting it for pronouncement now:
>>> https://www.python.org/dev/peps/pep-0493/
>>>
>>> There's currently no BDFL-Delegate assigned, so if Guido doesn't want
>>> to handle it, we'll need to address that question first.
>>>
>>> Our last discussion back in July seemed to show that folks either
>>> didn't care about the question (because they're using unmodified
>>> upstream versions so the PEP didn't affect them), or else thought the
>>> approach described in the PEP was reasonable, so I'm hoping the
>>> consequences of my mistake won't be too severe.
>>>
>>> Regards,
>>> Nick.
>>>
>>> P.S. I'm aware that this looks like presenting a fait accompli at a
>>> point where it's too late to realistically say "No", but the truth is
>>> that preparation for the Python in Education miniconf at PyCon
>>> Australia ramped up immediately after the July discussion, and then I
>>> personally got confused as to the scope of what was being included in
>>> 7.2 (I mistakenly thought it was just PEP 466 for now, with 476+493
>>> being deferred to a later release, but it's actually the whole package
>>> of 466+476+493). That's my fault for trying to keep track of too many
>>> things at once (and thus failing at some of them), not anyone else's.
>>>
>>> ================================
>>>
>>> PEP: 493
>>> Title: HTTPS verification recommendations for Python 2.7 redistributors
>>> Version: $Revision$
>>> Last-Modified: $Date$
>>> Author: Nick Coghlan <ncoghlan at gmail.com>,
>>>         Robert Kuska <rkuska at redhat.com>,
>>>         Marc-André Lemburg <mal at lemburg.com>
>>> Status: Draft
>>> Type: Informational
>>> Content-Type: text/x-rst
>>> Created: 10-May-2015
>>> Post-History: 06-Jul-2015
>>>
>>>
>>> Abstract
>>> ========
>>>
>>> PEP 476 updated Python's default handling of HTTPS certificates to be
>>> appropriate for communication over the public internet. The Python 2.7 long
>>> term maintenance series was judged to be in scope for this change, with the
>>> new behaviour introduced in the Python 2.7.9 maintenance release.
>>>
>>> This PEP provides recommendations to downstream redistributors wishing to
>>> provide a smoother migration experience when helping their users to manage
>>> this change in Python's default behaviour.
>>>
>>>
>>> Rationale
>>> =========
>>>
>>> PEP 476 changed Python's default behaviour to better match the needs and
>>> expectations of developers operating over the public internet, a category
>>> which appears to include most new Python developers. It is the position of
>>> the authors of this PEP that this was a correct decision.
>>>
>>> However, it is also the case that this change *does* cause problems for
>>> infrastructure administrators operating private intranets that rely on
>>> self-signed certificates, or otherwise encounter problems with the new default
>>> certificate verification settings.
>>>
>>> The long term answer for such environments is to update their internal
>>> certificate management to at least match the standards set by the public
>>> internet, but in the meantime, it is desirable to offer these administrators
>>> a way to continue receiving maintenance updates to the Python 2.7 series,
>>> without having to gate that on upgrades to their certificate management
>>> infrastructure.
>>>
>>> PEP 476 did attempt to address this question, by covering how to revert the
>>> new settings process wide by monkeypatching the ``ssl`` module to restore the
>>> old behaviour. Unfortunately, the ``sitecustomize.py`` based technique proposed
>>> to allow system administrators to disable the feature by default in their
>>> Standard Operating Environment definition has been determined to be
>>> insufficient in at least some cases. The specific case of interest to the
>>> authors of this PEP is the one where a Linux distributor aims to provide
>>> their users with a
>>> `smoother migration path
>>> <https://bugzilla.redhat.com/show_bug.cgi?id=1173041>`__
>>> than the standard one provided by consuming upstream CPython 2.7 releases
>>> directly, but other potential challenges have also been pointed out with
>>> updating embedded Python runtimes and other user level installations of Python.
>>>
>>> Rather than allowing a plethora of mutually incompatibile migration techniques
>>> to bloom, this PEP proposes two alternative approaches that redistributors
>>> may take when addressing these problems. Redistributors may choose to implement
>>> one, both, or neither of these approaches based on their assessment of the
>>> needs of their particular userbase.
>>>
>>> These designs are being proposed as a recommendation for redistributors, rather
>>> than as new upstream features, as they are needed purely to support legacy
>>> environments migrating from older versions of Python 2.7. Neither approach
>>> is being proposed as an upstream Python 2.7 feature, nor as a feature in any
>>> version of Python 3 (whether published directly by the Python Software
>>> Foundation or by a redistributor).
>>>
>>>
>>> Requirements for capability detection
>>> =====================================
>>>
>>> As these recommendations are intended to cover backports to earlier Python
>>> versions, the Python version number cannot be used as a reliable means for
>>> detecting them. Instead, the recommendations are defined to allow the presence
>>> or absence of the feature to be determined using the following technique::
>>>
>>>     python -c "import ssl; ssl._relevant_attribute"
>>>
>>> This will fail with `AttributeError` (and hence a non-zero return code) if the
>>> relevant capability is not available.
>>>
>>> The marker attributes are prefixed with an underscore to indicate the
>>> implementation dependent nature of these capabilities - not all Python
>>> distributions will offer them, only those that are providing a multi-stage
>>> migration process from the legacy HTTPS handling to the new default behaviour.
>>>
>>>
>>> Recommendation for an environment variable based security downgrade
>>> ===================================================================
>>>
>>> Some redistributors may wish to provide a per-application option to disable
>>> certificate verification in selected applications that run on or embed CPython
>>> without needing to modify the application itself.
>>>
>>> In these cases, a configuration mechanism is needed that provides:
>>>
>>> * an opt-out model that allows certificate verification to be selectively
>>>   turned off for particular applications after upgrading to a version of
>>>   Python that verifies certificates by default
>>> * the ability for all users to configure this setting on a per-application
>>>   basis, rather than on a per-system, or per-Python-installation basis
>>>
>>> This approach may be used for any redistributor provided version of Python 2.7,
>>> including those that advertise themselves as providing Python 2.7.9 or later.
>>>
>>>
>>> Required marker attribute
>>> -------------------------
>>>
>>> The required marker attribute on the ``ssl`` module when implementing this
>>> recommendation is::
>>>
>>>     _https_verify_envvar = 'PYTHONHTTPSVERIFY'
>>>
>>> This not only makes it straightforward to detect the presence (or absence) of
>>> the capability, it also makes it possible to programmatically determine the
>>> relevant environment variable name.
>>>
>>>
>>> Recommended modifications to the Python standard library
>>> --------------------------------------------------------
>>>
>>> The recommended approach to providing a per-application configuration setting
>>> for HTTPS certificate verification that doesn't require modifications to the
>>> application itself is to:
>>>
>>> * modify the ``ssl`` module to read the ``PYTHONHTTPSVERIFY`` environment
>>>   variable when the module is first imported into a Python process
>>> * set the ``ssl._create_default_https_context`` function to be an alias for
>>>   ``ssl._create_unverified_context`` if this environment variable is present
>>>   and set to ``'0'``
>>> * otherwise, set the ``ssl._create_default_https_context`` function to be an
>>>   alias for ``ssl.create_default_context`` as usual
>>>
>>>
>>> Example implementation
>>> ----------------------
>>>
>>> ::
>>>
>>>     _https_verify_envvar = 'PYTHONHTTPSVERIFY'
>>>
>>>     def _get_https_context_factory():
>>>         config_setting = os.environ.get(_https_verify_envvar)
>>>         if config_setting == '0':
>>>             return _create_unverified_context
>>>         return create_default_context
>>>
>>>     _create_default_https_context = _get_https_context_factory()
>>>
>>>
>>> Security Considerations
>>> -----------------------
>>>
>>> Relative to an unmodified version of CPython 2.7.9 or later, this approach
>>> does introduce a new downgrade attack against the default security settings
>>> that potentially allows a sufficiently determined attacker to revert Python
>>> to the vulnerable configuration used in CPython 2.7.8 and earlier releases.
>>> However, such an attack requires the ability to modify the execution
>>> environment of a Python process prior to the import of the ``ssl`` module,
>>> and any attacker with such access would already be able to modify the
>>> behaviour of the underlying OpenSSL implementation.
>>>
>>>
>>> Recommendation for backporting to earlier Python versions
>>> =========================================================
>>>
>>> Some redistributors, most notably Linux distributions, may choose to backport
>>> the PEP 476 HTTPS verification changes to modified Python versions based on
>>> earlier Python 2 maintenance releases. In these cases, a configuration
>>> mechanism is needed that provides:
>>>
>>> * an opt-in model that allows the decision to enable HTTPS certificate
>>>   verification to be made independently of the decision to upgrade to the
>>>   Python version where the feature was first backported
>>> * the ability for system administrators to set the default behaviour of Python
>>>   applications and scripts run directly in the system Python installation
>>> * the ability for the redistributor to consider changing the default behaviour
>>>   of *new* installations at some point in the future without impacting existing
>>>   installations that have been explicitly configured to skip verifying HTTPS
>>>   certificates by default
>>>
>>> This approach should not be used for any Python installation that advertises
>>> itself as providing Python 2.7.9 or later, as most Python users will have the
>>> reasonable expectation that all such environments will validate HTTPS
>>> certificates by default.
>>>
>>>
>>> Required marker attribute
>>> -------------------------
>>>
>>> The required marker attribute on the ``ssl`` module when implementing this
>>> recommendation is::
>>>
>>>     _cert_verification_config = '<path to configuration file>'
>>>
>>> This not only makes it straightforward to detect the presence (or absence) of
>>> the capability, it also makes it possible to programmatically determine the
>>> relevant configuration file name.
>>>
>>>
>>> Recommended modifications to the Python standard library
>>> --------------------------------------------------------
>>>
>>> The recommended approach to backporting the PEP 476 modifications to an earlier
>>> point release is to implement the following changes relative to the default
>>> PEP 476 behaviour implemented in Python 2.7.9+:
>>>
>>> * modify the ``ssl`` module to read a system wide configuration file when the
>>>   module is first imported into a Python process
>>> * define a platform default behaviour (either verifying or not verifying HTTPS
>>>   certificates) to be used if this configuration file is not present
>>> * support selection between the following three modes of operation:
>>>
>>>   * ensure HTTPS certificate verification is enabled
>>>   * ensure HTTPS certificate verification is disabled
>>>   * delegate the decision to the redistributor providing this Python version
>>>
>>> * set the ``ssl._create_default_https_context`` function to be an alias for
>>>   either ``ssl.create_default_context`` or ``ssl._create_unverified_context``
>>>   based on the given configuration setting.
>>>
>>>
>>> Recommended file location
>>> -------------------------
>>>
>>> This approach is currently only defined for \*nix system Python installations.
>>>
>>> The recommended configuration file name is
>>> ``/etc/python/cert-verification.cfg``.
>>>
>>> The ``.cfg`` filename extension is recommended for consistency with the
>>> ``pyvenv.cfg`` used by the ``venv`` module in Python 3's standard library.
>>>
>>>
>>> Recommended file format
>>> -----------------------
>>>
>>> The configuration file should use a ConfigParser ini-style format with a
>>> single section named ``[https]`` containing one required setting ``verify``.
>>>
>>> Permitted values for ``verify`` are:
>>>
>>> * ``enable``: ensure HTTPS certificate verification is enabled by default
>>> * ``disable``: ensure HTTPS certificate verification is disabled by default
>>> * ``platform_default``: delegate the decision to the redistributor providing
>>>   this particular Python version
>>>
>>> If the ``[https]`` section or the ``verify`` setting are missing, or if the
>>> ``verify`` setting is set to an unknown value, it should be treated as if the
>>> configuration file is not present.
>>>
>>>
>>> Example implementation
>>> ----------------------
>>>
>>> ::
>>>
>>>     _cert_verification_config = '/etc/python/cert-verification.cfg'
>>>
>>>     def _get_https_context_factory():
>>>         # Check for a system-wide override of the default behaviour
>>>         context_factories = {
>>>             'enable': create_default_context,
>>>             'disable': _create_unverified_context,
>>>             'platform_default': _create_unverified_context, # For now :)
>>>         }
>>>         import ConfigParser
>>>         config = ConfigParser.RawConfigParser()
>>>         config.read(_cert_verification_config)
>>>         try:
>>>             verify_mode = config.get('https', 'verify')
>>>         except (ConfigParser.NoSectionError, ConfigParser.NoOptionError):
>>>             verify_mode = 'platform_default'
>>>         default_factory = context_factories.get('platform_default')
>>>         return context_factories.get(verify_mode, default_factory)
>>>
>>>     _create_default_https_context = _get_https_context_factory()
>>>
>>>
>>> Security Considerations
>>> -----------------------
>>>
>>> The specific recommendations for the backporting case are designed to work for
>>> privileged, security sensitive processes, even those being run in the following
>>> locked down configuration:
>>>
>>> * run from a locked down administrator controlled directory rather than a normal
>>>   user directory (preventing ``sys.path[0]`` based privilege escalation attacks)
>>> * run using the ``-E`` switch (preventing ``PYTHON*`` environment variable based
>>>   privilege escalation attacks)
>>> * run using the ``-s`` switch (preventing user site directory based privilege
>>>   escalation attacks)
>>> * run using the ``-S`` switch (preventing ``sitecustomize`` based privilege
>>>   escalation attacks)
>>>
>>> The intent is that the *only* reason HTTPS verification should be getting
>>> turned off system wide when using this approach is because:
>>>
>>> * an end user is running a redistributor provided version of CPython rather
>>>   than running upstream CPython directly
>>> * that redistributor has decided to provide a smoother migration path to
>>>   verifying HTTPS certificates by default than that being provided by the
>>>   upstream project
>>> * either the redistributor or the local infrastructure administrator has
>>>   determined that it is appropriate to override the default upstream behaviour
>>>   (at least for the time being)
>>>
>>> Using an administrator controlled configuration file rather than an environment
>>> variable has the essential feature of providing a smoother migration path, even
>>> for applications being run with the ``-E`` switch.
>>>
>>>
>>> Combining the recommendations
>>> =============================
>>>
>>> If a redistributor chooses to implement both recommendations, then the
>>> environment variable should take precedence over the system-wide configuration
>>> setting. This allows the setting to be changed for a given user, virtual
>>> environment or application, regardless of the system-wide default behaviour.
>>>
>>> In this case, if the ``PYTHONHTTPSVERIFY`` environment variable is defined, and
>>> set to anything *other* than ``'0'``, then HTTPS certificate verification
>>> should be enabled.
>>>
>>> Example implementation
>>> ----------------------
>>>
>>> ::
>>>
>>>     _https_verify_envvar = 'PYTHONHTTPSVERIFY'
>>>     _cert_verification_config = '/etc/python/cert-verification.cfg'
>>>
>>>     def _get_https_context_factory():
>>>         # Check for am environmental override of the default behaviour
>>>         config_setting = os.environ.get(_https_verify_envvar)
>>>         if config_setting is not None:
>>>             if config_setting == '0':
>>>                 return _create_unverified_context
>>>             return create_default_context
>>>
>>>         # Check for a system-wide override of the default behaviour
>>>         context_factories = {
>>>             'enable': create_default_context,
>>>             'disable': _create_unverified_context,
>>>             'platform_default': _create_unverified_context, # For now :)
>>>         }
>>>         import ConfigParser
>>>         config = ConfigParser.RawConfigParser()
>>>         config.read(_cert_verification_config)
>>>         try:
>>>             verify_mode = config.get('https', 'verify')
>>>         except (ConfigParser.NoSectionError, ConfigParser.NoOptionError):
>>>             verify_mode = 'platform_default'
>>>         default_factory = context_factories.get('platform_default')
>>>         return context_factories.get(verify_mode, default_factory)
>>>
>>>     _create_default_https_context = _get_https_context_factory()
>>>
>>>
>>> Copyright
>>> =========
>>>
>>> This document has been placed into the public domain.
>>>
>>>
>>> --
>>> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>>> _______________________________________________
>>> Python-Dev mailing list
>>> Python-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>
>
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia



-- 
--Guido van Rossum (python.org/~guido)


More information about the Python-Dev mailing list