On 26 November 2015 at 06:17, Robert Collins firstname.lastname@example.org wrote:
On 26 November 2015 at 08:57, Barry Warsaw email@example.com wrote:
There's a lot to process in this thread, but as I see it, the issue breaks down to these questions:
How should PEP 493 be implemented?
What should the default be?
How should PEP 493 be worded to express the right tone to redistributors?
Let me take on the implementation details here.
On Nov 24, 2015, at 04:04 PM, M.-A. Lemburg wrote:
I would still find having built-in support for the recommendations in the Python stdlib a better approach
As would I.
For what its worth: a PEP telling distributors to patch the standard library is really distasteful to me.
I'm not a big fan of it either, but it's the way sustainable commercial open source distribution works in practice: customers want selective upgrades, so customers get selective upgrades; customers want greater platform integration, so customers get greater platform integration; etc, etc. Large Python shops like Google will typically maintain their own custom Python variants with their own patches applied that are suitable and sufficient for their use cases, but may not be suitable for pushing upstream (e.g. they may be Linux specific patches, without support for other platforms).
That's why I've focused on providing guidance to redistributors to keep their patches within reasonably consistent bounds for PEPs like bundling pip, backporting the network security enhancements to Python 2.7, and changing the default behaviour for HTTPS verification. Different redistributors have different customer bases, which also differ from the audience for upstream python.org releases, so saying "don't patch Python" is denying the reality of the obligations commercial vendors have to their customers, while "when you patch Python, please ensure you abide by these guidelines" is something redistributors can realistically do.
We've spent a long time trying to build close relations such that when something doesn't work distributors can share their needs with us and we can make Python out of the box be a good fit. This seems to fly in the exact opposite direction: we're explicitly making it so that Python builds on these vendor's platforms will not be the same as you get by checking out the Python source code.
This has always been true for the RHEL system Python - it gets backported changes, just like the kernel does (albeit nowhere near as many). That's one of the main things long term support customers are paying for - developers continuing to care about a code base after the volunteers working on the project upstream have (entirely reasonably) shifted their attentions to newer versions. We try to keep the patches to a minimum, and avoid having to carry large patches indefinitely by only backporting changes that have already been accepted in later versions, but "never apply patches to upstream projects as a downstream redistributor" isn't a realistic goal.
In this particular case, the migration problems were already raised in the PEP 476 discussions, and the decision was made to *not* provide a public API specifically for globally turning off HTTPS certificate verification, since that would either:
1. Need to be added to Python 3 and maintained indefinitely; or 2. Be implemented in Python 2.7 only, and thus create another barrier to 2->3 migration
A HTTPS specific flag wasn't considered a suitable long term API design for Python 3, while a multi-protocol configuration design that also handled setting the defaults for other protocols like secure SMTP, IMAP, POP3, SFTP, etc seemed generally desirable (but also doesn't exist anywhere, and poses signficant challenges in designing and implementing a good test matrix).
The only concrete things that have changed since PEP 476 was approved is that:
* we figured out the monkeypatching based migration recommendations in that PEP didn't actually work for the redistributor case * Robert Kuska came up with a way to do a backwards compatible backport of PEP 476 to RHEL 7.2 that doesn't raise any new barriers to migrating from Python 2 to Python 3
PEP 493 originally only described the latter idea - I added MAL's design for the environment variable based security downgrade based on the original PEP 493 discussions in July. Splitting the latter out to its own PEP that proposes it as a forwards compatible configuration setting akin to "curl --insecure" and "wget --no-check-certificate" for Python 2.7.12, 3.4.4, 3.5.2 and 3.6 upstream would be fine by me, since that would also significantly clarify the fact that the two ideas are entirely orthogonal to each other (except for the need to define a relative priority when you combine the two).