On Sep 2, 2014, at 7:47 PM, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:

On Sep 2, 2014, at 4:28 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:

On 3 Sep 2014 09:08, "David Reid" <dreid@dreid.org> wrote:
>
> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>
> > Creating *new* incompatibilities between Python 2 & Python 3 is a major point
> > of concern.
>
> Clearly this change should be backported to Python2.

Proposing to break backwards compatibility in a maintenance release (...)

As we keep saying, this is not a break in backwards compatibility, it's a bug fix.  Yes, systems might break, but that breakage represents an increase in security which may well be operationally important.  Not everyone with a "working" application has the relevant understanding an expertise to know that Python's HTTP client is exposing them to surveillance.  These applications should break. That is the very nature of the fix.  It is not a "compatibility break" that the system starts correctly rejecting invalid connections.

By way of analogy, here's another kind of breach in security: an arbitrary remote code execution vulnerability in XML-RPC.  I think we all agree that any 0day RCE vulnerabilities in Python really ought to be fixed and could be legitimately included without worrying about backwards compatibility breaks.  (At least... gosh, I hope so.)

Perhaps this arbitrary remote execution looks harmless; the use of an eval() instead of an int() someplace.  Perhaps someone discovered that they can do "3 + 4" in their XML-RPC and the server does the computation for them.  Great!  They start relying on this in their applications to use symbolic values in their requests instead of having explicit enumerations.  This can save you quite a bit of code!

When the RCE is fixed, this application will break, and that's fine.  In fact that's the whole point of issuing the fix, that people will no longer be able to make arbitrary computation requests of your server any more.  If that server's maintainer has the relevant context and actually wants the XML-RPC endpoint to enable arbitrary RCE, they can easily modify their application to start doing eval() on the data that they received, just as someone can easily modify their application to intentionally disable all connection security.  (Let's stop calling it "certificate verification" because that sounds like some kind of clerical detail: if you disable certificate verification, TLS connections are unauthenticated and unidentified and therefore insecure.)

For what it's worth, on the equivalent Twisted change, I originally had just these concerns, but my mind was changed when I considered what exactly the user-interface ramifications were for people typing that 's' for 'secure' in URLs.  I was convinced, and we made the change, and there have been no ill effects that I'm aware of as a result.  In fact, there has been a renewed interest in Twisted for HTTP client work, because we finally made security work more or less like it's supposed to, and the standard library is so broken.

I care about the health of the broader Python community, so I will passionately argue that this change should be made, but for me personally it's a lot easier to justify that everyone should use Twisted (at least since 14+) because transport security in the stdlib is such a wreck and even if it gets fixed it's going to have easy options to turn it off unilaterally so your application can never really be sure if it's getting transport security when it's requesting transport security.