[Python-Dev] PYTHONHTTPSVERIFY env var

[Nick Coghlan]

On 12 May 2015 at 21:45, Donald Stufft <donald at stufft.io> wrote:
>
>> On May 12, 2015, at 7:40 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>> On 12 May 2015 at 21:21, Donald Stufft <donald at stufft.io> wrote:
>>>
>>>> On May 12, 2015, at 7:17 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>>>
>>>> On 12 May 2015 at 21:09, Donald Stufft <donald at stufft.io> wrote:
>>>>> If you control the app you don't need to do that. All relevant api accept the context parameter. The shims are only useful when you don't control the app. So an app shipping their own python doesn't fall under that.
>>>>
>>>> I think the "bundled Python" scenario MAL is interested in is this one:
>>>>
>>>> 1. An application with a bundled CPython runtime is using the
>>>> verification defaults
>>>> 2. Upgraded the bundled Python to 2.7.9
>>>> 3. Didn't provide new configuration settings to disable certificate verification
>>>> 4. Is being upgraded in an environment where verifying certificates
>>>> makes the app unusable for environmental reasons related to
>>>> certificate management
>>>>
>>>> The PyRun single-file Python interpreter has a similar need, where
>>>> some apps than ran fine under 2.7.8 will need a way to disable cert
>>>> verification in 2.7.9+ on a per-application basis, *without* modifying
>>>> the applications.
>>>>
>>>> Both of those make sense to me as cases where the environment variable
>>>> based security downgrade approach is the "least bad" answer available,
>>>> which is why I eventually agreed it should be one of the
>>>> recommendations in the PEP.
>>>>
>>>
>>> Why is without modifying the app a reasonable goal? If Python is bundled
>>> with the app then you have direct control over when that upgrade happens,
>>> so you can delay the upgrade to 2.7.9 until your application which is
>>> bundling Python has the relevant switches. This is distinctly different
>>> from a situation like downstream distributors where the version of Python
>>> being provided is being provided by a group different than the person
>>> providing the application.
>>
>> Because of the way redistribution works. MAL was right that I was
>> thinking specifically in terms of the Linux distributor case, where
>> we're the OS vendor, so we need a way to offer "off by default, opt-in
>> on a per-server basis". Once I got past that perspective, I was able
>> to figure out where he was coming from as someone that offers explicit
>> support for the "redistribution for bundling" use case.
>>
>> When apps "bundle Python", what's usually happening is that they'll
>> just bundle whatever version is used on the build server that does the
>> bundling. If the app developer's testing all uses valid HTTPS
>> certificates (or simply doesn't test HTTPS at all), they won't see any
>> problems with the 2.7.9 upgrade, and hence will ship that along to
>> their customers, where it may break if that customer's environment
>> turns out to be relying on the lack of certificate verification in
>> 2.7.8 and earlier.
>>
>> If that scenario happens with unmodified upstream 2.7.9, the
>> redistributor has no workaround they can pass along to app developers
>> to in turn pass on to customers - the app developer simply has to tell
>> their customers to downgrade back to the previous release, and then
>> issue an updated version with a configuration setting to disable HTTPS
>> verification as fast as they can. Customers tend to get rather grouchy
>> about things like that :)
>>
>> By contrast, if the redistributor for the bundled version of Python
>> has injected PYTHONHTTPSVERIFY support, then the app developers can at
>> least pass along "set PYTHONHTTPSVERIFY=0 in the environment" to their
>> customers as an interim workaround until they get a release out the
>> door with a proper configuration setting to control whether or not the
>> app verifies certificates (assuming they don't decide the environment
>> variable is a good enough workaround).
>>
>
> Honestly, this reads like "If the person bundling 2.7.9 with their app doesn't
> bother to pay attention to what 2.7.9 means then things might break", but
> that's hardly special to TLS, there are lots of things that change in a release
> that may end up breaking in certain cases.

Cert verification is a special case, as cert management in intranets
tends to rely on two things:

* browser users just click through SSL security warnings
* automated internal tools don't check certs at all

Organisations with a specific interest in network security may have
their systems in a happier state, but I wouldn't bet on them being a
substantial minority of organisations, let alone a signficant
majority.

It's hard to overstate how big a mindset shift "getting intranet
network security right isn't optional" represents in our industry, and
it's going to take years for that attitude change to filter out
through the later parts of the technology adoption curve.

> Looking at Marc-Andre's latest email though, it appears we're using bundling
> in a different context? I'm thinking of things like PyInstaller or such where
> you're distributing Python + your own App, but this appears to just be some
> third party tool is installing Python and an App?

I'm personally talking about both, as for a lot of folks bundling
Python, the actual bundling process is handled by a third party tool,
and a lot of application vendors aren't going to think about what
happens where their app is run in an environment with bad certificate
management.

> Ultimately, as long as it doesn't end up in upstream CPython the PEP can
> recommend any approach and I'm OK with it in the sense that it won't affect me.
> Though the PEP should be clear it's for 2.7 only and not 3.x

Yep, I just pushed an update
(https://hg.python.org/peps/rev/b395246d0af7) that adds MAL's
environment variable based solution as an alternative recommendation,
and that update includes this phrase:

===================
These designs are being proposed as a recommendation for redistributors, rather
than as new upstream features, as they are needed purely to support legacy
environments migrating from older versions of Python 2.7. Neither approach
is being proposed as an upstream Python 2.7 feature, nor as a feature in any
version of Python 3 (whether published directly by the Python Software
Foundation or by a redistributor).
===================

> or 2.8 if that
> ever gets reversed and we end up with a 2.8.

There won't be a 2.8, so mentioning that these downstream
modifications wouldn't be used on a release that is never going to
happen would just confuse people :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia