On Jan 10, 2017, at 08:24 AM, Donald Stufft wrote:

python3 -c "import urllib.request,json; print(json.loads(urllib.request.urlopen('https://www.howsmyssl.com/a/check').read())['tls_version'])"

For Python 3.5: python3 -c "import urllib.request,json; print(json.loads(urllib.request.urlopen('https://www.howsmyssl.com/a/check').read().decode('utf-8'))['tls_version'])" Cheers, -Barry

On Jan 10, 2017, at 3:01 PM, Ronald Oussoren wrote:

...

On 10 Jan 2017, at 14:24, Donald Stufft mailto:donald@stufft.io> wrote: […] Past that, macOS is going to be the largest casualty since their system Python does not support TLSv1.2 yet in any version of their OS.

Not just the system Python on OSX, this also affects all Python.org http://python.org/ installers for OSX except 3.6. The 3.6 installer is the first one that doesn’t use the system installation of OpenSSL. Annoyingly with OpenSSL on OSX you have to options: either use an up-to-date release or have OpenSSL use the system CA trust store, but not both. Sigh…

I have no idea how may users use the Python.org http://python.org/ installers on OSX, but this has the potential to affect a largish number of users on OSX including newbies (but far from all users on OSX, there’s also a sizeable population using Homebrew or Anaconda).

Ronald

Ah yea I forgot those :( — Donald Stufft

On Jan 10, 2017, at 15:07, Ronald Oussoren wrote:

...

On 10 Jan 2017, at 21:02, Donald Stufft wrote:

...

On Jan 10, 2017, at 3:01 PM, Ronald Oussoren wrote:

...

On 10 Jan 2017, at 14:24, Donald Stufft wrote: […] Past that, macOS is going to be the largest casualty since their system Python does not support TLSv1.2 yet in any version of their OS. Not just the system Python on OSX, this also affects all Python.org installers for OSX except 3.6. The 3.6 installer is the first one that doesn’t use the system installation of OpenSSL.

That's not quite accurate. The 32-bit-only macOS python.org installers for recent 2.7.x and 3.x releases are also linked with a private current set of OpenSSL libraries. For 3.6, we no longer supply the 32-bit-only installer and the 64-bit/32-bit installer is now linked with the private OpenSSL as you note.

...

...

Annoyingly with OpenSSL on OSX you have to options: either use an up-to-date release or have OpenSSL use the system CA trust store, but not both. Sigh…

It would be nice if someone would do the work to figure out whether it is feasible to use Apple's own Crypto and TLS API's as apparently libcurl does.

...

...

I have no idea how may users use the Python.org installers on OSX, but this has the potential to affect a largish number of users on OSX including newbies (but far from all users on OSX, there’s also a sizeable population using Homebrew or Anaconda).

And MacPorts. I don't know about Anaconda but the other two already use their own private versions of OpenSSL AFAIK. -- Ned Deily nad@python.org -- []

On 10 January 2017 at 23:24, Donald Stufft wrote:

Looking at the download numbers, the absolute largest driver of TLSv1.0 and TLSv1.1 traffic to PyPI are old versions of pip or other clients where I cannot tell the OS that they are being run on.

Can you tell the Python version they're running even with older clients? I just checked the exact dates/versions where TLS v1.2 was properly enabled in the various versions of Python that Red Hat ships, and this change should be fine for: * RHEL/CentOS 7.2+ (PEP 466 backport released November 2015) * Red Hat Software Collections 2.2+ (PEP 466 backport released May 2016) However, folks currently using the system Python 2.6 installation in RHEL/CentOS 6 are going to need to upgrade to Python 2.7 somehow, whether that's by: - upgrading to RHEL/CentOS 7 - doing a parallel install via RHSCL/softwarecollections.org - doing a parallel install via ius.io (The problem with RHEL 6 is that even though the *OS* has supported TLS v1.2 since RHEL 6.5, *Python 2.6* doesn't properly support accessing them through the standard library's SSL module, since it's missing the features backported from 3.x by PEP 466) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 11 January 2017 at 14:04, Donald Stufft wrote:

On Jan 10, 2017, at 10:59 PM, Nick Coghlan wrote: (The problem with RHEL 6 is that even though the *OS* has supported TLS v1.2 since RHEL 6.5, *Python 2.6* doesn't properly support accessing them through the standard library's SSL module, since it's missing the features backported from 3.x by PEP 466)

No, but it doesn’t matter, the version of Python doesn’t control it at all since we use PROTOCOL_SSLv23 which will automatically negotiate the highest protocol OpenSSL supports, whether Python has bound the PROTOCOL_TLSv1_X constant and implemented the methods for it or not. So Python 2.6 is perfectly capable of talking to a TLSv1.2 site (it however, is not capable of explicitly saying it *needs* only TLSv1.2).

See:

$ python2.6 -c "import urllib2,json; print(json.loads(urllib2.urlopen('https://www.howsmyssl.com/a/check').read())['tls_version'])" TLS 1.2

Ah, excellent. In that case, RHEL 6 should be fine as well, as 6.5 was released back in 2013, and the extended update support for 6.4 ended in March 2015. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 1/11/2017 8:09 AM, Ronald Oussoren wrote:

...

On 10 Jan 2017, at 21:47, Ned Deily wrote:

...

...

...

Annoyingly with OpenSSL on OSX you have to options: either use an up-to-date release or have OpenSSL use the system CA trust store, but not both. Sigh…

It would be nice if someone would do the work to figure out whether it is feasible to use Apple's own Crypto and TLS API's as apparently libcurl does.

This should be possible, although I don’t know if the relevant Apple API’s are safe to use after forking (which has bitten us in the past for other Apple APIs, such as the _scproxy extension that detects proxies on OSX).

Sadly enough I don’t really have time to look into this.

While I'm sure this isn't a newbie "easy" issue, maybe open an issue for it on the bug tracker and someone will find it and work on it? I'd open it myself, but I don't know enough about it to phrase the problem correctly! Eric.

On 01/11/2017 10:26 AM, Brett Cannon wrote:

I know both Cory Benfield and Christian Heimes brought this up briefly at the PyCon US 2016 language summit at the end of their SSL discussion, but I don't think it went anywhere because there was some other discussion that dominated the end of their talk (I've now tweeted at them about this discussion).

I know Steve has also said he would love to see a agnostic TLS library so that Windows' built-in libraries for this stuff could be directly used. With the predicament this is going to put us in I think it makes it very prudent to create a tls module for the stdlib.

A thread about a new tls module has been started on the security-sig [1]. -- ~Ethan~ [1] security-sig@python.org https://mail.python.org/pipermail/security-sig/

On Jan 11, 2017, at 9:58 PM, Nick Coghlan wrote:

On 12 January 2017 at 04:26, Brett Cannon wrote:

...

On Tue, 10 Jan 2017 at 12:51 Donald Stufft wrote:

...

[SNIP]

It would be really nice if we could deprecate `ssl` (which has a bunch of OpenSSL specific stuff in it) and add a new `tls` module that served as an implementation agnostic library that would use OpenSSL on *nix, SecureTransport on macOS, and SChannel on Windows. However, in the mean time there are some folks poking to see about making something pip suitable that will enable us to use SecureTransport at least.

I know both Cory Benfield and Christian Heimes brought this up briefly at the PyCon US 2016 language summit at the end of their SSL discussion, but I don't think it went anywhere because there was some other discussion that dominated the end of their talk (I've now tweeted at them about this discussion).

I know Steve has also said he would love to see a agnostic TLS library so that Windows' built-in libraries for this stuff could be directly used. With the predicament this is going to put us in I think it makes it very prudent to create a tls module for the stdlib.

Logistically, something I think we should explore for such a module is using the same ensuretls/tls split that we did for ensurepip/pip. That way it can be more readily updated in line with the evolution of network security standards and operating system crpytographic APIs, rather than being locked into being updated in line with the evolution of the Python language definition.

This doesn’t work well because it’s not something that pip is going to be able to upgrade on Windows, because the .so will be locked when pip imports it on Windows and we won’t be able to uninstall it to do an upgrade. We had to disable the automatic use of pyOpenSSL for this reason too. The only C stuff that pip can reliably use is the standard library. — Donald Stufft

On Jan 11, 2017, at 10:40 PM, Nick Coghlan wrote:

On 12 January 2017 at 13:00, Donald Stufft wrote:

...

This doesn’t work well because it’s not something that pip is going to be able to upgrade on Windows, because the .so will be locked when pip imports it on Windows and we won’t be able to uninstall it to do an upgrade. We had to disable the automatic use of pyOpenSSL for this reason too. The only C stuff that pip can reliably use is the standard library.

Ugh, I'd completely forgotten about that limitation of Windows filesystems.

And the main alternatives I can think of involve copying files around as pip starts up, which would be unacceptably slow for a command line app :(

I don’t think it’s a particularly big deal to tie the tls module to the Python lifecycle though, we’ve got a precident for PEPs that backport important security critical stuff and most things are presumably going to be things that we don’t really even need a backport or a PEP for (I’m thinking things like ciphers and such). Particularly if this new thing is documented up front clearly what things you can depend on for compatibility (api and such) and what things can change in minor releases (keeping up with the security joneses stuff). I think the big thing that really killed the ssl module for so long in Python was the 2.x vs 3.x split with 2.7 living for a _very_ long time, and then no culture of back porting security important changes to it. — Donald Stufft

"I don’t think it’s a particularly big deal to tie the tls module to the Python lifecycle though" I'd hope that the API of this module is stable enough and the native part of the implementation is OS-specific enough that we may not even need to update it that often. (I'm advocating very strongly for just using the OS APIs to implement it, and those don't change often enough for us to need to worry.) The Linux builds can link to OpenSSL, but there shouldn't be anything requiring OpenSSL for this module, so the update timeframe is totally different. But I've now joined security-sig, which is where the discussion seems to be, so I'll stop designing things here :) Top-posted from my Windows Phone -----Original Message----- From: "Donald Stufft" Sent: ‎1/‎11/‎2017 19:48 To: "Nick Coghlan" Cc: "DistUtils mailing list" Subject: Re: [Distutils] Announcement: TLSv1.2 will become mandatory in thefuture On Jan 11, 2017, at 10:40 PM, Nick Coghlan wrote: On 12 January 2017 at 13:00, Donald Stufft wrote: This doesn’t work well because it’s not something that pip is going to be able to upgrade on Windows, because the .so will be locked when pip imports it on Windows and we won’t be able to uninstall it to do an upgrade. We had to disable the automatic use of pyOpenSSL for this reason too. The only C stuff that pip can reliably use is the standard library. Ugh, I'd completely forgotten about that limitation of Windows filesystems. And the main alternatives I can think of involve copying files around as pip starts up, which would be unacceptably slow for a command line app :( I don’t think it’s a particularly big deal to tie the tls module to the Python lifecycle though, we’ve got a precident for PEPs that backport important security critical stuff and most things are presumably going to be things that we don’t really even need a backport or a PEP for (I’m thinking things like ciphers and such). Particularly if this new thing is documented up front clearly what things you can depend on for compatibility (api and such) and what things can change in minor releases (keeping up with the security joneses stuff). I think the big thing that really killed the ssl module for so long in Python was the 2.x vs 3.x split with 2.7 living for a _very_ long time, and then no culture of back porting security important changes to it. — Donald Stufft

On 12 January 2017 at 14:12, Glyph Lefkowitz wrote:

It's possible for Pip to notice that it wants to replace a particular file; you can "unlock" it by moving it aside.

https://serverfault.com/a/503769

Very interesting, especially as the way CPython loads extension modules means that the renaming trick should be safe (even if you "reload" an extension module, or delete it from sys.modules and import it again, CPython doesn't reload the underlying DLL). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia