[Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Nick Coghlan]

On 27 November 2015 at 03:15, Barry Warsaw <barry at python.org> wrote:
> On Nov 26, 2015, at 03:06 PM, Nick Coghlan wrote:
>
>>I'm not a big fan of it either, but it's the way sustainable
>>commercial open source distribution works in practice:
>
> While it's inevitable that redistributors have to deviate from upstream, in
> Debian and Ubuntu, we really try to keep that at a minimum.  Debian, of course
> is an all volunteer organization so our "customers" are just normal users who
> I think want an experience as close to installing from source as possible,
> consistent with the principles and policies of the Debian project.  Debian at
> least tries to document the handful of deviations from upstream.

Yeah, we need to be better about that - while
https://access.redhat.com/articles/2039753 covers the details for the
certificate verification changes, there's currently no easy way to get
a description of the changes backported to the RHEL/CentOS system
Python without trawling through old Errata announcements :(

> Likewise in Ubuntu, we try to keep deviations from Debian at a minimum, and
> document them when we must deviate.  Ubuntu is a community driven distro so
> while Canonical itself has customers, it's much more likely that feedback
> about the Python stack comes from ordinary users.  Again, my personal goal is
> to make Python on Ubuntu a pleasant and comfortable environment, as close to
> installing from source as possible, consistent with the principles and
> policies of the project.

I'd strongly agree with that description for Fedora and
softwarecollections.org, but for the RHEL/CentOS system Python I think
the situation is slightly different: there, the goal is to meet the
long term support commitments involved in being a base RHEL package.
As the nominal base version of the package (2.7.5 in the case of RHEL
7) doesn't change, there is naturally going to be increasing
divergence from the nominal version.

> All this to say that IMHO, solutions to problems should go as far upstream as
> possible.  Guidelines certainly are useful (e.g. PEP 394) to maintain
> consistency across downstream providers where the decisions are primarily in
> the sphere of influence of the downstreams, but IMHO there's still more that
> upstream can do to promote PEP 493.

I tried to go down the "upstream first" path with a properly supported
"off switch" in PEP 476, and didn't succeed (hence the monkeypatch
compromise). It sounds like several folks would like to see us revisit
that decision, though.

>>Different redistributors have different customer bases, which also
>>differ from the audience for upstream python.org releases, so saying
>>"don't patch Python" is denying the reality of the obligations
>>commercial vendors have to their customers, while "when you patch
>>Python, please ensure you abide by these guidelines" is something
>>redistributors can realistically do.
>
> ISTM that the same forces are in play regardless of whether the change is in
> code or in an informational PEP.  Best to get consensus where possible, and
> manifest those decisions in code, but if competing goals are the outcome of a
> code change or informational PEP, downstream consumers will still make what
> they judge to be the best decision in the interest of their users, balanced
> against their own competing constraints.
>
>>In this particular case, the migration problems were already raised in
>>the PEP 476 discussions, and the decision was made to *not* provide a
>>public API specifically for globally turning off HTTPS certificate
>>verification, since that would either:
>>
>>1. Need to be added to Python 3 and maintained indefinitely; or
>>2. Be implemented in Python 2.7 only, and thus create another barrier
>>to 2->3 migration
>
> A minor one, for sure.  It wouldn't significantly bother me if the API were
> underscore-prefixed as a big honkin' clue about the guarantees being made.

OK, you've persuaded me to recast the PEP from an Informational one to
a Standards track one. The section about backporting to versions prior
to 2.7.9 won't change much (except in tone), but the ability to opt
out on a process-wide basis will be pitched as a language level
feature.

>>* Robert Kuska came up with a way to do a backwards compatible
>>backport of PEP 476 to RHEL 7.2 that doesn't raise any new barriers to
>>migrating from Python 2 to Python 3
>
> I'm not sure I agree with that.  If you have to set an environment variable to
> get your application to work in Python 2.7, it's *still* a migration issue
> when you move to Python 3.  You're just moving the change out of the code and
> into the environment, where it's arguably more difficult to re-discover down
> the road.

By that I meant that setting the environment variable won't *break*
anything in Python 3, whereas if we add a new Python level API and
people call it without a hasattr() guard, then even if their
infrastructure has been upgraded to use verifiable certificates, the
client applications will break on Python 3.

However, with an underscore prefix and an admonition to use a
hasattr() check, I agree a 2.7.x-only Python level API isn't really
any more of a barrier to migration than only offering the environment
variable.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia