On 31 August 2014 12:21, R. David Murray firstname.lastname@example.org wrote:
Do those knobs allow one to instruct urllib to accept an invalid certificate without changing the program code?
My first reply ended up being a context dump of the challenges created by legacy corporate intranets that may not be immediately obvious to folks that spend most of their time working on or with the public internet. I decided to split these more technical details out to a new reply for the benefit of folks that already know all that history :)
To answer David's specific question, the existing knobs at the OpenSSL level (SSL_CERT_DIR and SSL_CERT_FILE ) let people add an internal CA, opt out of the default CA system, and trust *specific* self-signed certs.
What they don't allow is a global "trust any cert" setting - exceptions need to be added at the individual cert level or at the CA level, or the application needs to offer an option to not do cert validation at all. That "trust anything" option at the platform level is the setting that is a really bad idea - if an organisation thinks it needs that (because they have a lot of self-signed certs, but aren't verifying their HTTPS connections to those servers), then what they really need is an internal CA, where their systems just need to be set up to trust the internal CA in addition to the platform CA certs.
With Alex's proposal, organisations that are already running an internal CA should be just fine - Python 3.5 will see the CA cert in the platform cert store and accept certs signed by it as valid. (Note: the Python 3.4 warning should take this into account, which could be a problem since we don't currently do validity checks against the platform store by default. The PEP needs to cover the mechanics of that in more detail, as I think it means we'll need to make *some* changes to the default configuration even in Python 3.4 to get accurate validity data back from OpenSSL)
However, we also need to accept that there's a reason browser vendors still offer "click through insecurity" for sites with self-signed certificates, and tools like wget/curl offer the option to say "don't check the certificate": these are necessary compromises to make SSL based network connections actually work on many current corporate intranets.
It is corporate environments that also make it desirable to be able to address this potential problem at a *user* level, since many Python users in a large organisations are actually running Python entirely out of their home directories, rather than as a system installation (they may not even have admin access to their own systems).
My suggestion at this point is that we take a leaf from both browser vendors and the design of SSH: make it easy to *add* a specific self-signed cert to the set a *particular user* trusts by default (preferably *only* for a particular host, to limit the power of such certs). "python -m ssl" doesn't currently do anything interesting, so it could be used to provide an API for managing that user level certificate store.
A Python-specific user level cert store is something that could be developed as a PyPI library for Python 2.7.9+ and 3.4+ (Is cert management considered in scope for cryptography.io? If so, that could be a good home).
So while I agree with the intent of PEP 476, and like the suggested end state, I'm back to thinking that the transition plan for existing corporate users needs more work before it can be accepted. This is especially true since it becomes another barrier to migrating from Python 2.7 to Python 3.5+ (a warning in Python 3.4 doesn't help with that aspect, although a new -3 warning might).
A third party module that offers a user level certificate store, and a gevent.monkey style way of opting in to this behaviour for existing Python versions would be one way to provide a more compelling transition plan.