Withdraw PEP 546? Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7
Hi, tl; dr I will withdraw the PEP 546 in one week if noboy shows up to finish the implementation. Last year,I wrote the PEP 546 with Cory Benfield: "Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7" https://www.python.org/dev/peps/pep-0546/ The plan was to get a Python 2.7 implementation of Cory's PEP 543: "A Unified TLS API for Python" https://www.python.org/dev/peps/pep-0543/ Sadly, it seems like Cory is no longer available to work on the projec (PEP 543 is still a draft)t. The PEP 546 is implemented: https://github.com/python/cpython/pull/2133 Well, I closed it, but you can still get it as a patch with: https://patch-diff.githubusercontent.com/raw/python/cpython/pull/2133.patch But tests fail on Travis CI whereas I'm unable to reproduce the issue on my laptop (on Fedora). The failure seems to depend on the version of OpenSSL. Christian Heimes has a "multissl" tool which automates tests on multiple OpenSSL versions, but I failed to find time to try this tool. Time flies and one year later, the PR of the PEP 546 is still not merged, tests are still failing. One month ago, when 2.7.15 has been released, Benjamin Peterson, Python 2.7 release manager, simply proposed: "The lack of movement for a year makes me wonder if PEP 546 should be moved to Withdrawn status." Since again, I failed to find time to look at the test_ssl failure, I plan to withdraw the PEP next week if nobody shows up :-( Sorry Python 2.7! Does anyone would benefit of MemoryBIO in Python 2.7? Twisted, asyncio, trio, urllib3, anyone else? If yes, who is volunteer to finish the MemoryBIO backport (and maintain it)? Victor
On Wed, May 30, 2018, 07:30 Victor Stinner <vstinner@redhat.com> wrote:
Does anyone would benefit of MemoryBIO in Python 2.7? Twisted, asyncio, trio, urllib3, anyone else?
Asyncio and trio are strongly py3-only. Twisted's TLS functionality is built around pyopenssl, so the stdlib ssl module doesn't affect them. Urllib3 uses the socket-wrapping APIs, not MemoryBIO. So fwiw I don't think any of those projects would benefit. -n
2018-05-30 18:02 GMT+02:00 Nathaniel Smith <njs@pobox.com>:
On Wed, May 30, 2018, 07:30 Victor Stinner <vstinner@redhat.com> wrote:
Does anyone would benefit of MemoryBIO in Python 2.7? Twisted, asyncio, trio, urllib3, anyone else?
Asyncio and trio are strongly py3-only. Twisted's TLS functionality is built around pyopenssl, so the stdlib ssl module doesn't affect them. Urllib3 uses the socket-wrapping APIs, not MemoryBIO. So fwiw I don't think any of those projects would benefit.
MemoryBIO was the key feature which allowed to implement TLS for the ProactorEventLoop (IOCP) of asyncio. I'm not sure that the Python 2.7 ssl module is a drop-in replacement for pyopenssl. Victor
On Wed, May 30, 2018, 14:21 Victor Stinner <vstinner@redhat.com> wrote:
On Wed, May 30, 2018, 07:30 Victor Stinner <vstinner@redhat.com> wrote:
Does anyone would benefit of MemoryBIO in Python 2.7? Twisted, asyncio, trio, urllib3, anyone else?
Asyncio and trio are strongly py3-only. Twisted's TLS functionality is built around pyopenssl, so the stdlib ssl module doesn't affect them. Urllib3 uses the socket-wrapping APIs, not MemoryBIO. So fwiw I don't think any of
2018-05-30 18:02 GMT+02:00 Nathaniel Smith <njs@pobox.com>: those
projects would benefit.
MemoryBIO was the key feature which allowed to implement TLS for the ProactorEventLoop (IOCP) of asyncio.
MemoryBIO is definitely super useful for async libraries – trio uses it, asyncio uses it, twisted uses it (via pyopenssl). But I don't know of anyone who currently needs it but hasn't already found a way to get it. I'm not sure that the Python 2.7 ssl module is a drop-in replacement
for pyopenssl.
No, their APIs are totally different, for better or worse. -n
On 31 May 2018 at 19:34, Nathaniel Smith <njs@pobox.com> wrote:
On Wed, May 30, 2018, 14:21 Victor Stinner <vstinner@redhat.com> wrote:
MemoryBIO was the key feature which allowed to implement TLS for the ProactorEventLoop (IOCP) of asyncio.
MemoryBIO is definitely super useful for async libraries – trio uses it, asyncio uses it, twisted uses it (via pyopenssl). But I don't know of anyone who currently needs it but hasn't already found a way to get it.
I think one of the other key things that changed is pip gaining its own native support for using the SecureTransport API on Mac OS X. So yeah, unless someone from PyCA chimes in to say that the PEP still offers benefits that we can't get another way, withdrawing PEP 546 as "Overtaken by events" probably makes sense. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On May 31, 2018, at 10:22 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 31 May 2018 at 19:34, Nathaniel Smith <njs@pobox.com <mailto:njs@pobox.com>> wrote: On Wed, May 30, 2018, 14:21 Victor Stinner <vstinner@redhat.com <mailto:vstinner@redhat.com>> wrote: MemoryBIO was the key feature which allowed to implement TLS for the ProactorEventLoop (IOCP) of asyncio.
MemoryBIO is definitely super useful for async libraries – trio uses it, asyncio uses it, twisted uses it (via pyopenssl). But I don't know of anyone who currently needs it but hasn't already found a way to get it.
I think one of the other key things that changed is pip gaining its own native support for using the SecureTransport API on Mac OS X.
So yeah, unless someone from PyCA chimes in to say that the PEP still offers benefits that we can't get another way, withdrawing PEP 546 as "Overtaken by events" probably makes sense.
I think it still provides benefits FWIW, pip’s SecureTransport shim is a slow as hell ctypes hack that is slow enough we *only* use it when the ``ssl`` library wouldn’t be able to connect anyways and that doesn’t help Windows. OTOH I don’t think it’s super useful without PEP 543 also.
+1 for withdrawing it. It's much too late in the 2.7 maintenance schedule to start bothering with such a large and perilous backport. Regards Antoine. On Wed, 30 May 2018 16:28:22 +0200 Victor Stinner <vstinner@redhat.com> wrote:
Hi,
tl; dr I will withdraw the PEP 546 in one week if noboy shows up to finish the implementation.
Last year,I wrote the PEP 546 with Cory Benfield: "Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7" https://www.python.org/dev/peps/pep-0546/
The plan was to get a Python 2.7 implementation of Cory's PEP 543: "A Unified TLS API for Python" https://www.python.org/dev/peps/pep-0543/
Sadly, it seems like Cory is no longer available to work on the projec (PEP 543 is still a draft)t.
The PEP 546 is implemented: https://github.com/python/cpython/pull/2133
Well, I closed it, but you can still get it as a patch with: https://patch-diff.githubusercontent.com/raw/python/cpython/pull/2133.patch
But tests fail on Travis CI whereas I'm unable to reproduce the issue on my laptop (on Fedora). The failure seems to depend on the version of OpenSSL. Christian Heimes has a "multissl" tool which automates tests on multiple OpenSSL versions, but I failed to find time to try this tool.
Time flies and one year later, the PR of the PEP 546 is still not merged, tests are still failing.
One month ago, when 2.7.15 has been released, Benjamin Peterson, Python 2.7 release manager, simply proposed: "The lack of movement for a year makes me wonder if PEP 546 should be moved to Withdrawn status."
Since again, I failed to find time to look at the test_ssl failure, I plan to withdraw the PEP next week if nobody shows up :-( Sorry Python 2.7!
Does anyone would benefit of MemoryBIO in Python 2.7? Twisted, asyncio, trio, urllib3, anyone else? If yes, who is volunteer to finish the MemoryBIO backport (and maintain it)?
Victor
So sorry for Python 2.7, I just rejected my PEP 546, no ssl.MemoryBIO for you! https://www.python.org/dev/peps/pep-0546/#rejection-notice The workaround is to use PyOpenSSL on Python 2.7. Victor 2018-05-30 16:28 GMT+02:00 Victor Stinner <vstinner@redhat.com>:
Hi,
tl; dr I will withdraw the PEP 546 in one week if noboy shows up to finish the implementation.
Last year,I wrote the PEP 546 with Cory Benfield: "Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7" https://www.python.org/dev/peps/pep-0546/
The plan was to get a Python 2.7 implementation of Cory's PEP 543: "A Unified TLS API for Python" https://www.python.org/dev/peps/pep-0543/
Sadly, it seems like Cory is no longer available to work on the projec (PEP 543 is still a draft)t.
The PEP 546 is implemented: https://github.com/python/cpython/pull/2133
Well, I closed it, but you can still get it as a patch with: https://patch-diff.githubusercontent.com/raw/python/cpython/pull/2133.patch
But tests fail on Travis CI whereas I'm unable to reproduce the issue on my laptop (on Fedora). The failure seems to depend on the version of OpenSSL. Christian Heimes has a "multissl" tool which automates tests on multiple OpenSSL versions, but I failed to find time to try this tool.
Time flies and one year later, the PR of the PEP 546 is still not merged, tests are still failing.
One month ago, when 2.7.15 has been released, Benjamin Peterson, Python 2.7 release manager, simply proposed: "The lack of movement for a year makes me wonder if PEP 546 should be moved to Withdrawn status."
Since again, I failed to find time to look at the test_ssl failure, I plan to withdraw the PEP next week if nobody shows up :-( Sorry Python 2.7!
Does anyone would benefit of MemoryBIO in Python 2.7? Twisted, asyncio, trio, urllib3, anyone else? If yes, who is volunteer to finish the MemoryBIO backport (and maintain it)?
Victor
participants (5)
-
Antoine Pitrou -
Donald Stufft -
Nathaniel Smith -
Nick Coghlan -
Victor Stinner