On 2017-01-30 14:53, David Cournapeau wrote:
Hi,
I am managing the team responsible for providing python packaging at Enthought, and I would like to make sure we are providing a good (and secure) out of the box experience for SSL.
My understanding is that PEP 476 is the latest PEP that concerns this issue, and that PEP recommends using the system store: https://www.python.org/dev/peps/pep-0476/#trust-database. But looking at binary python distributions from python.org <http://python.org>, that does not seem to a.ways be the case. I looked at the following:
* 3.5.3 from python.org <http://python.org> for OS X (64 bits): this uses the old, system openssl
On macOS, system OpenSSL uses TEA to look up trust anchors from the keyring, https://hynek.me/articles/apple-openssl-verification-surprises/
* 3.6.0 from python.org <http://python.org> for OS X: this embeds a recent openssl, but ssl seems to be configured to use non existing paths (ssl..get_default_verify_paths()), and indeed, cert validation seems to fail by default with those installers
I guess you haven't read the warning on the download page, https://www.python.org/downloads/release/python-360/ :) tl;dr Python on macOS can no longer use the keyring for certificates. You have to install certifi. Personally I (as a maintainer of the ssl module) do neither like certifi nor the approach taken. It's wrong and dangerous because certifi by-passes system policies and does not include local CAs. But it's the best we got.
* 3.6.0 from python.org <http://python.org> for windows: I have not found how the ssl module finds the certificate, but certification seems to work
The magic happens in load_default_certs(), https://hg.python.org/cpython/file/tip/Lib/ssl.py#l445 . The approach does work most of the time. Sometimes it fails because Windows download root CA's on demand. The code does not trigger a download. Steve and I have some ideas to improve the situation, https://bugs.python.org/issue28747 .
Are there any official recommendations for downstream packagers beyond PEP 476 ? Is it "acceptable" for downstream packagers to patch python's default cert locations ?
My semi-official recommendations are: * On Linux and BSD, use OpenSSL from your vendor (Red Hat, SuSE, Ubuntu, Debian, Arch, Gentoo, ...). Vendors compile OpenSSL with special flags to make get_default_verify_paths() work. * If your vendor prefers LibreSSL, you might run into minor issue. LibreSSL is not fully compatible to OpenSSL. Your vendor also compiles LibresSL with special flags. * Windows kinda works. We are going to improve the situation for 3.7. * On macOS ... well, please contact Apple service and demand a solution. Cory Benfield's TLS ABC PEP and Paul Kehrer's work on SecureTransport bindings for Python will let you use Apple's native TLS lib (with some restrictions regarding fork()). In the mean time you are screwed and have to rely on certifi. As last resort, use certifi. You HAVE to make sure to keep it up to date and you have to deal with local policies and CAs yourself. Christian