<div dir="ltr"><div class="gmail_quote"><div dir="ltr"><div>I wanted to use pyOpenSSL to verify a certificate chain in a TLS Certificate Handshake message, and in the process stumbled upon a _disturbing_ amount of misinformation online about how to do this, both using openSSL and with pyOpenSSL.</div><div><br></div><div>This isn't about a bug in pyOpenSSL, but the combination of a) lack of documentation and b) potentially misleading unit tests could result in serious vulnerabilities for users of the library.</div><div><br></div><div>It's clear from what I've found online that developers are confused and may have introduced vulnerabilities into their code.</div><div><br></div><div>I sent this to the pyOpenSSL security contact and was cleared to post this to the list to solicit feedback.</div><div><br></div><div>FWIW there seems to be an ongoing effort to _properly_ support chain verification in pyOpenSSL:</div><div><br></div><div><a href="https://github.com/pyca/pyopenssl/issues/502" target="_blank">https://github.com/pyca/<wbr>pyopenssl/issues/502</a></div><div><a href="https://github.com/pyca/pyopenssl/pull/473" target="_blank">https://github.com/pyca/<wbr>pyopenssl/pull/473</a></div><div><br></div><div>But earlier changes to pyOpenSSL claim to support some form of chain verification, and appear to be the source of confusion for developers:</div><div><br></div><div><a href="https://github.com/pyca/pyopenssl/pull/155" target="_blank">https://github.com/pyca/<wbr>pyopenssl/pull/155</a></div><div><br></div><div>The rest is long-winded but captures some of the issues with existing documentation online as well as the pyOpenSSL verify_certificate() function for those interested.</div><div><br></div><div>best wishes</div><div>Joel</div><div><br></div><div><br></div><div>Cert "Chain" Verification</div><div>-------------------------</div><div><br></div><div>As part of a system I'm building, I'd like to independently verify the server certificate included in a TLS Certificate (0x0b) Handshake message, sent by the server after the ServerHello Handshake message.</div><div><br></div><div>The Certificate Handshake message contains the server's certificate, intermediate certificates needed to verify the server certificate, and (optionally) the root certificate of the CA that issued the intermediate cert(s).</div><div><br></div><div>From the TLS RFC: <a href="https://tools.ietf.org/html/rfc5246#page-47" target="_blank">https://tools.ietf.org/html/<wbr>rfc5246#page-47</a></div><div><br></div><div>certificate_list</div><div>This is a sequence (chain) of certificates. The sender's</div><div>certificate MUST come first in the list. Each following</div><div>certificate MUST directly certify the one preceding it. Because</div><div>certificate validation requires that root keys be distributed</div><div>independently, the self-signed certificate that specifies the root</div><div>certificate authority MAY be omitted from the chain, under the</div><div>assumption that the remote end must already possess it in order to</div><div>validate it in any case.</div><div><br></div><div>Lets frame the problem: My goal is to verify this chain against a root certificate that I already know about and trust. The intermediates I'm presented are certificates I may never have seen before, and may have been tampered with in transit or replaced by an attacker that is impersonating the server I'm trying to connect to. Thus _chain_ is an important term here - I need to verify the site certificate, but in the process, I need to check the validity of any intermediate certs as well. In this use case, the intermediate cert is UNTRUSTED data and nothing prevents an attacker from replacing it with a self-signed certificate (e.g. root cert) - Calling it an intermediate because thats what the server claims it is doesn't make it one.</div><div><br></div><div>To me, this is what verifying a chain means. And IMHO, this is the 'common case' for using intermediates - there are contexts where a system directly chooses to trust an intermediate, but I do not.</div><div><br></div><div>Unfortunately, many of the solutions posted online (including those for pyOpenSSL) mistakenly trust the intermediate and make bypassing the verification process trivial for my use case.</div><div><br></div><div><br></div><div>The Command Line</div><div>----------------</div><div><br></div><div>Lets review some of the misinformation about the use of 'openssl verify' for this purpose, as this misinformation could be the source of future bugs if developers decide to re-implement verification based on the command-line behavior. How do we supply the chain cert(s) for our use case? The intermediates are untrusted, so the answer is to use -untrusted (check man verify) but this is not the only suggestion. Googling turns up harmful guidance on the issue. For example, the top hit:</div><div><br></div><div><a href="http://stackoverflow.com/questions/25482199/verify-a-certificate-chain-using-openssl-verify" target="_blank">http://stackoverflow.com/<wbr>questions/25482199/verify-a-<wbr>certificate-chain-using-<wbr>openssl-verify</a></div><div><br></div><div>This thread fortunately mentions -untrusted, but also includes several contradictory recommendations*, including the following to verify UserCert.pem:</div><div><br></div><div>openssl verify -verbose -CAfile <(cat Intermediate.pem RootCert.pem) UserCert.pem</div><div><br></div><div>Here, the intermediate and root are passed via the -CAfile command line argument, and the certificate to verify is in UserCert.pem. Is the poster deliberately trusting the intermediate? A person asks 'Will this actually verify the intermediate cert against the root cert?' and someone responds saying yes, it does.</div><div><br></div><div>Variations of this advice is repeated elsewhere in various forms, for example:</div><div><br></div><div><a href="http://superuser.com/questions/904859/why-cant-i-verify-this-certificate-chain" target="_blank">http://superuser.com/<wbr>questions/904859/why-cant-i-<wbr>verify-this-certificate-chain</a></div><div><br></div><div>recommending:</div><div><br></div><div>$ cat root.pem intermediate.pem > concat.pem</div><div>$ openssl verify -CAfile concat.pem john.pem</div><div>john.pem: OK</div><div><br></div><div>This discussion also seems to echo the above, depending on how enduser-example.com.chain is generated:</div><div><br></div><div><a href="https://raymii.org/s/tutorials/OpenSSL_command_line_Root_and_Intermediate_CA_including_OCSP_CRL%20and_revocation.html" target="_blank">https://raymii.org/s/<wbr>tutorials/OpenSSL_command_<wbr>line_Root_and_Intermediate_CA_<wbr>including_OCSP_CRL%20and_<wbr>revocation.html</a></div><div><br></div><div>openssl verify -CAfile enduser-certs/enduser-example.<wbr>com.chain enduser-certs/enduser-example.<wbr>com.crt</div><div><br></div><div>So what happens if we follow this advice for our problem?</div><div><br></div><div><br></div><div><div>-CAfile vs -untrusted</div><div>---------------------</div><div><br></div><div>Lets compare the approach described above to the use of -untrusted with real certificates. When we connect to <a href="http://sometechcompany.com" target="_blank">sometechcompany.com</a> we get the server certificate sometechcompany.com.pem, the intermediate sometechcompany_ist_ca_2.pem, and the CA certificate geotrust_global_ca.pem. *After confirming that geotrust_global_ca.pem is a certificate we trust*, we use openssl verify to verify the chain (note that the RFC allows the root CA to be omitted):</div><div><br></div><div>Both of these indicate that that the sometechcompany.com.pem is trusted:</div><div><br></div><div>$ openssl verify -CAfile geotrust_global_ca.pem -untrusted sometechcompany_ist_ca_2.pem sometechcompany.com.pem</div><div>$ openssl verify -CAfile <(cat geotrust_global_ca.pem sometechcompany_ist_ca_2.pem) sometechcompany.com.pem</div><div><br></div><div>If we omit the chain cert, or replace it with an invalid chain certificate, verification fails (as it should):</div><div><br></div><div>$ verify -CAfile <(cat geotrust_global_ca.pem) sometechcompany.com.pem</div><div>sometechcompany.com.pem: CN = <a href="http://sometechcompany.com" target="_blank">sometechcompany.com</a>, OU = management:idms.group.105316, O = sometechcompany Inc., ST = California, C = US</div><div>error 20 at 0 depth lookup:unable to get local issuer certificate</div><div><br></div><div>$ verify -CAfile <(cat geotrust_global_ca.pem invalid_chain_cert.crt) sometechcompany.com.pem</div><div>sometechcompany.com.pem: CN = <a href="http://sometechcompany.com" target="_blank">sometechcompany.com</a>, OU = management:idms.group.105316, O = sometechcompany Inc., ST = California, C = US</div><div>error 20 at 0 depth lookup:unable to get local issuer certificate</div><div><br></div><div>It's complaining about the missing chain cert, so it seems like it must be validating it against the root CA. These work too:</div><div><br></div><div>$ openssl verify -CAfile <(cat sometechcompany_ist_ca_2.pem) sometechcompany.com.pem</div><div>sometechcompany.com.pem: OK</div><div><br></div><div>$ openssl verify -untrusted sometechcompany_ist_ca_2.pem sometechcompany.com.pem</div><div>sometechcompany.com.pem: OK</div><div><br></div><div>So on this system, openssl is using the trusted certificates for verification. Because sometechcompany_ist_ca_2.pem isn't self signed, this *is* actually "verifying" it.</div><div><br></div><div>Unfortunately, an "intermediate" cert that is actually a root / self-signed _will be treated as a trusted CA_ when using the recommended command given above:</div><div><br></div><div>$ openssl verify -CAfile <(cat geotrust_global_ca.pem rogue_ca.pem) fake_sometechcompany_from_<wbr>rogue_ca.com.pem</div><div>fake_sometechcompany_from_<wbr>rogue_ca.com.pem: OK</div><div><br></div><div>This makes it trivial for a malicious server to "impersonate" <a href="http://sometechcompany.com" target="_blank">sometechcompany.com</a>, at least from the point of view of our verification scheme. The attacker creates a root cert, sign fake_sometechcompany_from_<wbr>rogue_ca.com.pem with it, and sends this root as a so-called "intermediate". As we've seen, this approach to verification will show the server cert as valid, when in fact it is not issued by any of the CAs we trust.</div><div><br></div><div>To go back to the right solution: Specifying the intermediate as untrusted (via -untrusted) causes this ultimately-untrusted certificate to fail validation, as it should:</div><div><br></div><div>$ openssl verify -CAfile ~/geotrust_global_ca.pem -untrusted rogue_ca.pem fake_sometechcompany_from_<wbr>rogue_ca.com.pem</div><div>fake_sometechcompany_from_<wbr>rogue_ca.com.pem: C = US, ST = Vermont, L = Barre, O = Trusted CA Intermediate Signing Cert, CN = <a href="http://intermediate.trusted.ca" target="_blank">intermediate.trusted.ca</a>, emailAddress = <a href="mailto:intermediate@yahoo.com" target="_blank">intermediate@yahoo.com</a></div><div>error 19 at 1 depth lookup:self signed certificate in certificate chain</div><div><br></div><div>A rogue intermediate produces the same results.</div><div><br></div><div>Note that there are more subtleties to using verify (w.r.t. purpose), also discussed <a href="http://stackoverflow.com/questions/23304139/openssl-verify-gives-ok-for-bad-certificate-chain" target="_blank">http://stackoverflow.com/<wbr>questions/23304139/openssl-<wbr>verify-gives-ok-for-bad-<wbr>certificate-chain</a></div><div><br></div><div>This link also points out another gruesome usability wart for the command line tool - openSSL ends up ignoring multiple certs in the file passed for verification, and only verifies the first cert (I saw this recommended somewhere but can't find the link).</div><div><br></div><div><br></div><div>Chain verification in pyopenssl:</div><div>------------------------------<wbr>--</div><div><br></div><div>This is all a diversion though - My real goal was to use pyOpenSSL for this purpose. There's nothing in the documentation about doing this so looking at the unit tests and code (grepping chain) is the next step. I'm not the first to do so - Googling 'how to verify certificate chain in python' produces the following stackoverflow post as the first hit. The poster bases their recommendations on pyopenssl unit tests:</div><div><br></div><div><a href="http://stackoverflow.com/questions/30700348/how-to-validate-verify-an-x509-certificate-chain-of-trust-in-python" target="_blank">http://stackoverflow.com/<wbr>questions/30700348/how-to-<wbr>validate-verify-an-x509-<wbr>certificate-chain-of-trust-in-<wbr>python</a></div><div><br></div><div>Another hit, same recommendation:</div><div><br></div><div><a href="http://www.yothenberg.com/validate-x509-certificate-in-python/" target="_blank">http://www.yothenberg.com/<wbr>validate-x509-certificate-in-<wbr>python/</a></div><div><br></div><div>This second at least calls the intermediate cert 'trusted', but that contradicts the idea of any 'chain' of verification, at least in the sense of the term for my use case.</div><div><br></div><div>The recommended approach from these links, drawn from the unit tests, boils down to the following when verifying server_cert:</div><div><br></div><div>store = X509Store()</div><div>store.add_cert(root_cert)</div><div>store.add_cert(intermediate_<wbr>cert)</div><div>store_ctx = X509StoreContext(store, server_cert)</div><div><br></div><div>We see this code makes no distinction between the root_cert and the intermediate. If we look at the documentation, add_cert itself adds a *trusted* cert (maybe add_trusted_cert would be a better name?). So this is looking like the command-line example discussed above:</div><div><br></div><div>openssl verify -verbose -CAfile <(cat Intermediate.pem RootCert.pem) UserCert.pem</div></div><div><br></div><div><br></div><div><div>How NOT to do chain verification</div><div>------------------------------<wbr>--</div><div><br></div><div>Lets try it, using the same certs used above:</div><div><br></div><div>from OpenSSL.crypto import load_certificate, load_privatekey, FILETYPE_PEM</div><div>from OpenSSL.crypto import X509Store, X509StoreContext</div><div>from six import u, b, binary_type, PY3</div><div><br></div><div>root_cert_pem = open('geotrust_global_ca.pem')<wbr>.read()</div><div>intermediate_cert_pem = open('sometechcompany_ist_ca_<wbr>2.pem').read()</div><div>intermediate_server_cert_pem = open('sometechcompany.com.pem'<wbr>).read()</div><div><br></div><div>rogue_ca_cert_pem = open('rogue_ca.pem').read()</div><div>fake_server_cert_from_rogue_<wbr>ca_pem = open('fake_sometechcompany_<wbr>from_rogue_ca.com.pem').read()</div><div><br></div><div>root_cert = load_certificate(FILETYPE_PEM, root_cert_pem)</div><div>intermediate_cert = load_certificate(FILETYPE_PEM, intermediate_cert_pem)</div><div>intermediate_server_cert = load_certificate(FILETYPE_PEM, intermediate_server_cert_pem)</div><div>rogue_ca_cert = load_certificate(FILETYPE_PEM, rogue_ca_cert_pem)</div><div>fake_server_cert_from_rogue_ca = load_certificate(FILETYPE_PEM, fake_server_cert_from_rogue_<wbr>ca_pem)</div><div><br></div><div># recommended approach from the links above, prints None:</div><div><br></div><div>store = X509Store()</div><div>store.add_cert(root_cert)</div><div>store.add_cert(intermediate_<wbr>cert)</div><div>store_ctx = X509StoreContext(store, intermediate_server_cert)</div><div><br></div><div>print(store_ctx.verify_<wbr>certificate())</div><div><br></div><div># BAD - attacker substitutes a rogue CA in place of the intermediate, this prints None:</div><div><br></div><div>store = X509Store()</div><div>store.add_cert(root_cert)</div><div>store.add_cert(rogue_ca_cert)</div><div>store_ctx = X509StoreContext(store, fake_server_cert_from_rogue_<wbr>ca)</div><div><br></div><div>print(store_ctx.verify_<wbr>certificate())</div><div><br></div><div># interestingly, this causes an exception to be raised, and thus the behavior is a bit better than the 'openssl verify' antipattern shown above:</div><div><br></div><div>store = X509Store()</div><div>store.add_cert(intermediate_<wbr>cert)</div><div>store_ctx = X509StoreContext(store, intermediate_server_cert)</div><div><br></div><div>print(store_ctx.verify_<wbr>certificate())</div><div><br></div><div>Again, This makes it trivial for a malicious server to "impersonate" <a href="http://sometechcompany.com" target="_blank">sometechcompany.com</a> and bypass our verification. As we saw, the attacker creates a root cert, sign fake_sometechcompany_from_<wbr>rogue_ca.com.pem with it, and sends this root as a so-called "intermediate".</div><div><br></div><div><br></div><div>Unit tests</div><div>----------</div><div><br></div><div>For completeness, here is one of the unit tests referenced by the stackoverflow post - here, as we mentioned, add_cert adds a *trusted* intermediate certificate. This is reflected in the documentation for that function, but not clear from usage, hence the confusion above.</div><div><br></div><div>3644 def test_valid(self):</div><div>3645 """</div><div>3646 :py:obj:`verify_certificate` returns ``None`` when called with a</div><div>3647 certificate and valid chain.</div><div>3648 """</div><div>3649 store = X509Store()</div><div>3650 store.add_cert(self.root_cert)</div><div>3651 store.add_cert(self.<wbr>intermediate_cert)</div><div>3652 store_ctx = X509StoreContext(store, self.intermediate_server_cert)</div><div>3653 self.assertEqual(store_ctx.<wbr>verify_certificate(), None)</div></div><div><br></div><div><br></div><div><div>an explicitly flawed verify_chain()</div><div>------------------------------<wbr>-----</div><div><br></div><div>The ability to verify certificate chains in pyopenssl was originally discussed in the following issue (can be found using google):</div><div><br></div><div><a href="https://github.com/pyca/pyopenssl/issues/154" target="_blank">https://github.com/pyca/<wbr>pyopenssl/issues/154</a></div><div><br></div><div>and first contributed in the following change which explicitly uses this dangerous antipattern of trusting the chain certs:</div><div><br></div><div><a href="https://github.com/sholsapp/pyopenssl/blob/f8b517803ecb812a2140e45e069b20e0ec1c0389/OpenSSL/crypto.py" target="_blank">https://github.com/sholsapp/<wbr>pyopenssl/blob/<wbr>f8b517803ecb812a2140e45e069b20<wbr>e0ec1c0389/OpenSSL/crypto.py</a></div><div><br></div><div>2307 def verify_chain(cert, trust, chain=None):</div><div>2308 """</div><div>2309 Verify a chain of certificates.</div><div>2310</div><div>2311 :param cert: certificate to verify</div><div>2312 :param trust: certificate to trust</div><div>2313 :param chain: additional certificates needed to verify the chain</div><div>2314 :return: None if the chain is valid, raise exception otherwise</div><div>2315 """</div><div>2316 store = X509Store()</div><div>2317 if chain:</div><div>2318 for _cert in chain:</div><div>2319 store.add_cert(_cert)</div><div>2320 store.add_cert(trust)</div><div>2321</div><div>2322 store_ctx = _lib.X509_STORE_CTX_new()</div><div>2323 _lib.X509_STORE_CTX_init(<wbr>store_ctx, store._store, cert._x509, _ffi.NULL);</div><div>2324</div><div>2325 ret = _lib.X509_verify_cert(store_<wbr>ctx)</div><div>2326 if ret <= 0:</div><div>2327 _raise_current_error()</div><div><br></div><div>These changes were ultimately abandoned and verify_certificate() was added here:</div><div><br></div><div><a href="https://github.com/pyca/pyopenssl/pull/155" target="_blank">https://github.com/pyca/<wbr>pyopenssl/pull/155</a></div><div><br></div><div><br></div><div>"untrusted" certs</div><div>-----------------</div><div><br></div><div>There seem to be some references to "untrusted" certificates in open issues:</div><div><br></div><div><a href="https://github.com/pyca/pyopenssl/issues/502" target="_blank">https://github.com/pyca/<wbr>pyopenssl/issues/502</a></div><div><br></div><div>And these changes seem to add support for chain certificates in the sense I described in my original use case:</div><div><br></div><div><a href="https://github.com/pyca/pyopenssl/pull/473" target="_blank">https://github.com/pyca/<wbr>pyopenssl/pull/473</a></div><div><br></div><div>It will be good to call out the earlier changes mentioned above in these issues so folks are warned against using the existing verify_certificate.</div><div><br></div><div><br></div><div>* Strangly, a comment in that thread actually recommends _against_ using untrusted: "-untrusted doesn't check whether certificate chain is fully valid. Please consider to pass both intermediate and root to command as -CAfile as other questions suggests. – Envek Jun 22 at 18:06"</div></div></div>
</div><br></div>