On Jun 2, 2017 7:24 AM, "Ben Darnell" <ben@bendarnell.com> wrote:
The PEP's rationale is now "This PEP will help facilitate the future adoption
of :pep:`543` across all supported Python versions, which will improve security
for both Python 2 and Python 3 users."

What exactly are these security improvements? My understanding (which may well be incorrect) is that the security improvements come in the future when the PEP 543 APIs are implemented on top of the various platform-native security libraries. These changes will not be present until some future 3.x release, and will not be backported to 2.7 (without another PEP, which I expect would be even more contentious than this one). What then is the security improvement for Python 2 users?

My understanding is that PEP 543 would be released as a library on pypi, as well as targeting stdlib inclusion in some future release. The goal of the MemoryBIO PEP is to allow the PEP 543 package to be pure-python, support all the major platforms, and straddle py2/py3


In Tornado, I have not felt any urgency to replace wrap_socket with MemoryBIO. Is there a security-related reason I should do so sooner rather than later? (I'd also urge Cory and any other wrap_socket skeptics on the requests team to reconsider - Tornado's SSLIOStream works well. The asynchronous use of wrap_socket isn't so subtle and risky with buffering)

I'll leave the discussion of wrap_socket's reliability to others, because I don't have any experience there, but I do want to point out that that which primitive you pick has major system design consequences. MemoryBIO is an abstraction that can implement wrap_socket, but not vice-versa; if you use wrap_socket as your fundamental primitive then that leaks all over your abstraction hierarchy.

Twisted has to have a MemoryBIO​ implementation of their TLS code, because they want to support TLS over arbitrary transports. Carrying a wrap_socket implementation as well would mean twice the tricky and security-sensitive code to maintain, plus breaking their current abstractions to expose whether any particular transport object is actually just a raw socket.

The problem gets worse when you add PEP 543's pluggable TLS backends. If you can require a MemoryBIO-like API, then adding a backend becomes a matter of defining like 4 methods with relatively simple, testable interfaces, and it automatically works everywhere. Implementing a wrap_socket interface on top of this is complicated because the socket API is complicated and full of subtle corners, but it only has to be done once and libraries like tornado can adapt to the quirks of the one implementation.

OTOH if you have to also support backends that only have wrap_socket, then this multiplies the complexity of everything. Now we need a way to negotiate which APIs each backend supports, and we need to somehow document all the weird corners of how wrapped sockets are supposed to behave in edge cases, and when different wrapped sockets inevitably behave differently then libraries like tornado need to discover those differences and support all the options. We need a way to explain to users why some backends work with some libraries but not others. And so on. And we still need to support MemoryBIOs in as many cases as possible.

-n