[Python-Dev] PEP 476: Enabling certificate validation by default!

[Donald Stufft]

> On Aug 29, 2014, at 5:58 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> 
> On 29.08.2014 23:11, Donald Stufft wrote:
>> 
>> Sorry I was on my phone and didn’t get to fully reply to this.
>> 
>>> On Aug 29, 2014, at 4:00 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>>> 
>>> On 29.08.2014 21:47, Alex Gaynor wrote:
>>>> Hi all,
>>>> 
>>>> I've just submitted PEP 476, on enabling certificate validation by default for
>>>> HTTPS clients in Python. Please have a look and let me know what you think.
>>>> 
>>>> PEP text follows.
>>> 
>>> Thanks for the PEP. I think this is generally a good idea,
>>> but some important parts are missing from the PEP:
>>> 
>>> * transition plan:
>>> 
>>>  I think starting with warnings in Python 3.5 and going
>>>  for exceptions in 3.6 would make a good transition
>>> 
>>>  Going straight for exceptions in 3.5 is not in line with
>>>  our normal procedures for backwards incompatible changes.
>> 
>> As far as a transition plan, I think that this is an important
>> enough thing to have an accelerated process. If we need
>> to provide a warning than let’s add it to the next 3.4 otherwise
>> it’s going to be 2.5+ years until we stop being unsafe by
>> default.
> 
> Fine with me; we're still early in the Python 3.4
> patch level releases.
> 
>> Another problem with this is that I don’t think it’s actually
>> possible to do. Python itself isn’t validating the TLS certificates,
>> OpenSSL is doing that. To my knowledge OpenSSL doesn’t
>> have a way to say “please validate these certificates and if
>> they don’t validate go ahead and keep going and just let me
>> get a warning from it”. It’s a 3 way switch, no validation, validation
>> if a certificate is provided, and validation always.
>> 
>> Now that’s strictly for the “verify the certificate chain” portion,
>> the hostname verification is done entirely on our end and we
>> could do something there… but I’m not sure it makes sense
>> to do so if we can’t do it for invalid certificates too.
> 
> OpenSSL provides a callback for certificate validation,
> so it is possible to issue a warning and continue with
> accepting the certificate.

Ah right, I forgot about that. I was thinking in terms of CERT_NONE,
CERT_OPTIONAL, CERT_REQUIRED. I think it’s fine to add a warning
if possible to Python 3.4, I just couldn’t think off the top of my head
a way of doing it.

> 
>>> * configuration:
>>> 
>>>  It would be good to be able to switch this on or off
>>>  without having to change the code, e.g. via a command
>>>  line switch and environment variable; perhaps even
>>>  controlling whether or not to raise an exception or
>>>  warning.
>> 
>> I’m on the fence about this, if someone provides a certificate
>> that we can validate against (which can be done without
>> touching the code) then the only thing that really can’t be
>> “fixed” without touching the code is if someone has a certificate
>> that is otherwise invalid (expired, not yet valid, wrong hostname,
>> etc). I’d say if I was voting on this particular thing I’d be -0, I’d
>> rather it didn’t exist but I wouldn’t cry too much if it did.
> 
> If you're testing code or trying out some new stuff, you
> don't want to get a valid cert first, but instead go ahead
> with a self signed one. That's the use case.
> 
>>> * choice of trusted certificate:
>>> 
>>>  Instead of hard wiring using the system CA roots into
>>>  Python it would be good to just make this default and
>>>  permit the user to point Python to a different set of
>>>  CA roots.
>>> 
>>>  This would enable using self signed certs more easily.
>>>  Since these are often used for tests, demos and education,
>>>  I think it's important to allow having more control of
>>>  the trusted certs.
>> 
>> 
>> Like my other email said, the Python API has everything needed
>> to easily specify your own CA roots and/or disable the validations.
>> The OpenSSL library also allows you to specify either a directory
>> or a file to change the root certificates without code changes. The
>> only real problems with the APIs are that the default is bad and
>> an unrelated thing where you can’t pass in an in memory certificate.
> 
> Are you sure that's possible ? Python doesn't load the
> openssl.cnf file and the SSL_CERT_FILE, SSL_CERT_DIR env
> vars only work for the openssl command line binary, AFAIK.

I’m not 100% sure on that. I know they are not limited to the command
line binary as ruby uses those environment variables in the way I
described above. I do not believe that Ruby has done anything
special to enable the use of those variables. It’s possible we’re doing
something differently that bypasses those variables though. If that is the
case then yes let’s add it, ideally doing whatever it needs to be to make
OpenSSL respect those variables, or else respecting them ourselves.

> 
> In any case, Python will have to tap into the OS CA root
> provider using special code and this code could easily be
> made to check other dirs/files as well.
> 
> The point is that it should be possible to change this default
> at the Python level, without needing application code changes.

Ok, I’m not opposed to it FWIW. Just sayiing I’m pretty sure those things
already exist in the form of environment variables and at the python level
APIs. Not sure what else there is, global state for the “default”? A CLI
flag?

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140829/3d23c029/attachment-0001.html>