[Python-Dev] PYTHONHTTPSVERIFY env var

[M.-A. Lemburg]

On 11.05.2015 12:15, Nick Coghlan wrote:
> On 11 May 2015 at 19:22, M.-A. Lemburg <mal at egenix.com> wrote:
>> On 11.05.2015 11:13, Nick Coghlan wrote:
>>> I wouldn't be opposed to seeing that as an upstream Python 2.7.x
>>> feature, but agreement amongst redistributors on using a file-based
>>> approach is the main outcome I'm looking for.
>>
>> Can't we have both ?
> 
> I'd strongly advise against it, as we're deliberately increasing the
> attack surface area here by providing a potential path to carry out a
> downgrade attack on the HTTPS certificate verification by forcing it
> back to the old behaviour.
> 
> The main existing environment variable based attack vector would be to
> manage to run a process with administrative privileges and
> SSL_CERT_DIR and/or SSL_CERT_FILE pointing to a certificate written to
> a user or world-writable directory by the attacker. Providing a "don't
> verify HTTPS" flag at the interpreter level would let an attacker skip
> the first step of writing the certificate file to disk somewhere,
> making a system compromise harder to detect. (An especially paranoid
> SSL implementation would disallow reading certs from locations that
> allow write access to non-administrative users, but I don't believe
> OpenSSL is that paranoid)

Correct. OpenSSL will happily read the cert files from anywhere
you point it to.

> By contrast, the configuration file shouldn't provide a new attack
> vector (or simplify any existing attack vector), as if you have the
> permissions needed to modify the config file, you likely also have the
> permissions needed to modify the system certificate store, and the
> latter is a *far* more interesting attack vector than a downgrade
> attack solely against Python.
> 
> Thus the environment variable based off switch is neither necessary
> (as an administrator controlled configuration file can do the job),
> nor sufficient (it can't handle the -E switch), *and* it represents an
> increase in the attack surface area relative to a Python
> implementation without the capability.

Whether or not -E will have an effect on the env var depends
on the implementation. At the moment, -E only has an effect
on the C runtime, while the stdlib happily reads from os.environ
without taking the flag into account.

As proposed, the PYTHONHTTPSVERIFY would only affect the ssl
module and only be checked when loading this module, i.e. not
at Python startup time.

>> I don't think that we can wait for a whole PEP process to
>> run through
> 
> Matrix multiplication made it through the PEP process in 8 days. If we
> do this as a redistributor recommendation rather than attempting to
> get it into upstream Python 2.7, we could potentially propose you take
> on the role of BDFL-Delegate and mark it as Accepted as soon as the
> two of us agree on a common approach.
> 
> The reason I think that's a reasonable way forward is because we
> already know there are folks opposed to making the change upstream. If
> the PEP just provides recommendations for redistributors that *do*
> decide to provide a "global off switch" to revert to the old
> behaviour, then the perspective of the folks opposed to the feature is
> respected by the fact that this is a feature some redistributors *may*
> choose to add to provide a smoother migration path to more secure
> default HTTPS handling, rather than something upstream provides by
> default.
> 
> I assume the Debian, Ubuntu and Suse folks won't care, as they have
> all already decided against backporting the change to their long term
> support releases where the compatibility break would pose a problem
> (and I can certainly sympathise with that perspective given the
> dependency on backporting the PEP 466 SSL changes first, and the work
> involved in seeking consensus on a smoother migration path from the
> old default to the new one).
> 
> It would be nice to hear from ActiveState, Enthought & Continuum
> Analytics as well, but if they don't chime in here, I don't see any
> particular need to go chasing them explicitly.

I think the approach to only consider a subset of redistributors
as viable targets for such a switch is a bit too narrow.

You are leaving out all the parties which use custom
Python installations to run their applications, e.g.
the Plone and Zope community, the ZenOSS community,
the many Windows applications built on Python, etc.

>> to fix this regression in 2.7.9.
> 
> We made the decision when PEP 476 was accepted that this change turned
> a silent security failure into a noisy one, rather than being a
> regression in its own right. PEP 493 isn't about disagreeing with that
> decision, it's about providing a smoother upgrade path in contexts
> where letting the security failure remain silent is deemed to be
> preferred in the near term.

The change wasn't regression. The missing downgrade path
is a regression.

Some other comments on PEP 493:

* I don't think we really want to add the overhead of
  having to parse an INI file every time Python starts up.
  Please remember that we just parsing of the sysconfig
  data not long ago because we wanted to avoid this startup
  time.

* I don't see why the attack surface of using an INI file
  somewhere in the system should be smaller than e.g. using
  sitecustomize.py

* If done right, we'd also need a switch to ignore this
  global config file and recommend using it to reduce the
  attack surface (for the same reason you explain in the
  PEP)

* I don't think a global switch is the right way forward.
  Many applications on properly configured systems will
  work fine with the new default. The downgrade option is
  only needed for those cases, where they don't and you
  don't have a good way to fix the application.

* Most applications use some kind of virtualenv
  Python environment to run the code. These are
  typically isolated from the system Python installation
  and so wouldn't want to use a system wide global INI
  neither.

* The -S switch completely disables importing site.py.
  That's not really a viable solution in the age of
  pip - your local installation wouldn't find the installed
  packages anymore, since these are installed in site-packages/
  which again, is set up by site.py.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 11 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/