[Cryptography-dev] on how (not) to chain certs with openssl + pyopenssl

Joel Sandin jsandin at gmail.com
Mon Aug 22 12:16:59 EDT 2016

I wanted to use pyOpenSSL to verify a certificate chain in a TLS
Certificate Handshake message, and in the process stumbled upon a
_disturbing_ amount of misinformation online about how to do this, both
using openSSL and with pyOpenSSL.

This isn't about a bug in pyOpenSSL, but the combination of a) lack of
documentation and b) potentially misleading unit tests could result in
serious vulnerabilities for users of the library.

It's clear from what I've found online that developers are confused and may
have introduced vulnerabilities into their code.

I sent this to the pyOpenSSL security contact and was cleared to post this
to the list to solicit feedback.

FWIW there seems to be an ongoing effort to _properly_ support chain
verification in pyOpenSSL:


But earlier changes to pyOpenSSL claim to support some form of chain
verification, and appear to be the source of confusion for developers:


The rest is long-winded but captures some of the issues with existing
documentation online as well as the pyOpenSSL verify_certificate() function
for those interested.

best wishes

Cert "Chain" Verification

As part of a system I'm building, I'd like to independently verify the
server certificate included in a TLS Certificate (0x0b) Handshake message,
sent by the server after the ServerHello Handshake message.

The Certificate Handshake message contains the server's certificate,
intermediate certificates needed to verify the server certificate, and
(optionally) the root certificate of the CA that issued the intermediate

>From the TLS RFC: https://tools.ietf.org/html/rfc5246#page-47

This is a sequence (chain) of certificates.  The sender's
certificate MUST come first in the list.  Each following
certificate MUST directly certify the one preceding it.  Because
certificate validation requires that root keys be distributed
independently, the self-signed certificate that specifies the root
certificate authority MAY be omitted from the chain, under the
assumption that the remote end must already possess it in order to
validate it in any case.

Lets frame the problem: My goal is to verify this chain against a root
certificate that I already know about and trust.  The intermediates I'm
presented are certificates I may never have seen before, and may have been
tampered with in transit or replaced by an attacker that is impersonating
the server I'm trying to connect to.  Thus _chain_ is an important term
here - I need to verify the site certificate, but in the process, I need to
check the validity of any intermediate certs as well.  In this use case,
the intermediate cert is UNTRUSTED data and nothing prevents an attacker
from replacing it with a self-signed certificate (e.g. root cert) - Calling
it an intermediate because thats what the server claims it is doesn't make
it one.

To me, this is what verifying a chain means.  And IMHO, this is the 'common
case' for using intermediates - there are contexts where a system directly
chooses to trust an intermediate, but I do not.

Unfortunately, many of the solutions posted online (including those for
pyOpenSSL) mistakenly trust the intermediate and make bypassing the
verification process trivial for my use case.

The Command Line

Lets review some of the misinformation about the use of 'openssl verify'
for this purpose, as this misinformation could be the source of future bugs
if developers decide to re-implement verification based on the command-line
behavior.  How do we supply the chain cert(s) for our use case?  The
intermediates are untrusted, so the answer is to use -untrusted (check man
verify) but this is not the only suggestion.  Googling turns up harmful
guidance on the issue.  For example, the top hit:


This thread fortunately mentions -untrusted, but also includes several
contradictory recommendations*, including the following to verify

openssl verify -verbose -CAfile <(cat Intermediate.pem RootCert.pem)

Here, the intermediate and root are passed via the -CAfile command line
argument, and the certificate to verify is in UserCert.pem.  Is the poster
deliberately trusting the intermediate?  A person asks 'Will this actually
verify the intermediate cert against the root cert?' and someone responds
saying yes, it does.

Variations of this advice is repeated elsewhere in various forms, for



$ cat root.pem intermediate.pem > concat.pem
$ openssl verify -CAfile concat.pem john.pem
john.pem: OK

This discussion also seems to echo the above, depending on how
enduser-example.com.chain is generated:


openssl verify -CAfile enduser-certs/enduser-example.com.chain

So what happens if we follow this advice for our problem?

-CAfile vs -untrusted

Lets compare the approach described above to the use of -untrusted with
real certificates.  When we connect to sometechcompany.com we get the
server certificate sometechcompany.com.pem, the intermediate
sometechcompany_ist_ca_2.pem, and the CA certificate
geotrust_global_ca.pem.  *After confirming that geotrust_global_ca.pem is a
certificate we trust*, we use openssl verify to verify the chain (note that
the RFC allows the root CA to be omitted):

Both of these indicate that that the sometechcompany.com.pem is trusted:

$ openssl verify -CAfile geotrust_global_ca.pem -untrusted
sometechcompany_ist_ca_2.pem sometechcompany.com.pem
$ openssl verify -CAfile <(cat geotrust_global_ca.pem
sometechcompany_ist_ca_2.pem) sometechcompany.com.pem

If we omit the chain cert, or replace it with an invalid chain certificate,
verification fails (as it should):

$ verify -CAfile <(cat geotrust_global_ca.pem) sometechcompany.com.pem
sometechcompany.com.pem: CN = sometechcompany.com, OU =
management:idms.group.105316, O = sometechcompany Inc., ST = California, C
= US
error 20 at 0 depth lookup:unable to get local issuer certificate

$ verify -CAfile <(cat geotrust_global_ca.pem invalid_chain_cert.crt)
sometechcompany.com.pem: CN = sometechcompany.com, OU =
management:idms.group.105316, O = sometechcompany Inc., ST = California, C
= US
error 20 at 0 depth lookup:unable to get local issuer certificate

It's complaining about the missing chain cert, so it seems like it must be
validating it against the root CA.  These work too:

$ openssl verify -CAfile <(cat sometechcompany_ist_ca_2.pem)
sometechcompany.com.pem: OK

$ openssl verify -untrusted sometechcompany_ist_ca_2.pem
sometechcompany.com.pem: OK

So on this system, openssl is using the trusted certificates for
verification.  Because sometechcompany_ist_ca_2.pem isn't self signed, this
*is* actually "verifying" it.

Unfortunately, an "intermediate" cert that is actually a root / self-signed
_will be treated as a trusted CA_ when using the recommended command given

$ openssl verify -CAfile <(cat geotrust_global_ca.pem rogue_ca.pem)
fake_sometechcompany_from_rogue_ca.com.pem: OK

This makes it trivial for a malicious server to "impersonate"
sometechcompany.com, at least from the point of view of our verification
scheme.  The attacker creates a root cert, sign
with it, and sends this root as a so-called "intermediate".  As we've seen,
this approach to verification will show the server cert as valid, when in
fact it is not issued by any of the CAs we trust.

To go back to the right solution: Specifying the intermediate as untrusted
(via -untrusted) causes this ultimately-untrusted certificate to fail
validation, as it should:

$ openssl verify -CAfile ~/geotrust_global_ca.pem -untrusted rogue_ca.pem
fake_sometechcompany_from_rogue_ca.com.pem: C = US, ST = Vermont, L =
Barre, O = Trusted CA Intermediate Signing Cert, CN =
intermediate.trusted.ca, emailAddress = intermediate at yahoo.com
error 19 at 1 depth lookup:self signed certificate in certificate chain

A rogue intermediate produces the same results.

Note that there are more subtleties to using verify (w.r.t. purpose), also
discussed http://stackoverflow.com/questions/23304139/openssl-

This link also points out another gruesome usability wart for the command
line tool - openSSL ends up ignoring multiple certs in the file passed for
verification, and only verifies the first cert (I saw this recommended
somewhere but can't find the link).

Chain verification in pyopenssl:

This is all a diversion though - My real goal was to use pyOpenSSL for this
purpose.  There's nothing in the documentation about doing this so looking
at the unit tests and code (grepping chain) is the next step.  I'm not the
first to do so - Googling 'how to verify certificate chain in python'
produces the following stackoverflow post as the first hit.  The poster
bases their recommendations on pyopenssl unit tests:


Another hit, same recommendation:


This second at least calls the intermediate cert 'trusted', but that
contradicts the idea of any 'chain' of verification, at least in the sense
of the term for my use case.

The recommended approach from these links, drawn from the unit tests, boils
down to the following when verifying server_cert:

store = X509Store()
store_ctx = X509StoreContext(store, server_cert)

We see this code makes no distinction between the root_cert and the
intermediate.  If we look at the documentation, add_cert itself adds a
*trusted* cert (maybe add_trusted_cert would be a better name?).  So this
is looking like the command-line example discussed above:

openssl verify -verbose -CAfile <(cat Intermediate.pem RootCert.pem)

How NOT to do chain verification

Lets try it, using the same certs used above:

from OpenSSL.crypto import load_certificate, load_privatekey, FILETYPE_PEM
from OpenSSL.crypto import X509Store, X509StoreContext
from six import u, b, binary_type, PY3

root_cert_pem = open('geotrust_global_ca.pem').read()
intermediate_cert_pem = open('sometechcompany_ist_ca_2.pem').read()
intermediate_server_cert_pem = open('sometechcompany.com.pem').read()

rogue_ca_cert_pem = open('rogue_ca.pem').read()
fake_server_cert_from_rogue_ca_pem = open('fake_sometechcompany_

root_cert = load_certificate(FILETYPE_PEM, root_cert_pem)
intermediate_cert = load_certificate(FILETYPE_PEM, intermediate_cert_pem)
intermediate_server_cert = load_certificate(FILETYPE_PEM,
rogue_ca_cert = load_certificate(FILETYPE_PEM, rogue_ca_cert_pem)
fake_server_cert_from_rogue_ca = load_certificate(FILETYPE_PEM,

# recommended approach from the links above, prints None:

store = X509Store()
store_ctx = X509StoreContext(store, intermediate_server_cert)


# BAD - attacker substitutes a rogue CA in place of the intermediate, this
prints None:

store = X509Store()
store_ctx = X509StoreContext(store, fake_server_cert_from_rogue_ca)


# interestingly, this causes an exception to be raised, and thus the
behavior is a bit better than the 'openssl verify' antipattern shown above:

store = X509Store()
store_ctx = X509StoreContext(store, intermediate_server_cert)


Again, This makes it trivial for a malicious server to "impersonate"
sometechcompany.com and bypass our verification.  As we saw, the attacker
creates a root cert, sign fake_sometechcompany_from_rogue_ca.com.pem with
it, and sends this root as a so-called "intermediate".

Unit tests

For completeness, here is one of the unit tests referenced by the
stackoverflow post - here, as we mentioned, add_cert adds a *trusted*
intermediate certificate.  This is reflected in the documentation for that
function, but not clear from usage, hence the confusion above.

3644     def test_valid(self):
3645         """
3646         :py:obj:`verify_certificate` returns ``None`` when called with
3647         certificate and valid chain.
3648         """
3649         store = X509Store()
3650         store.add_cert(self.root_cert)
3651         store.add_cert(self.intermediate_cert)
3652         store_ctx = X509StoreContext(store,
3653         self.assertEqual(store_ctx.verify_certificate(), None)

an explicitly flawed verify_chain()

The ability to verify certificate chains in pyopenssl was originally
discussed in the following issue (can be found using google):


and first contributed in the following change which explicitly uses this
dangerous antipattern of trusting the chain certs:


2307 def verify_chain(cert, trust, chain=None):
2308     """
2309     Verify a chain of certificates.
2311     :param cert: certificate to verify
2312     :param trust: certificate to trust
2313     :param chain: additional certificates needed to verify the chain
2314     :return: None if the chain is valid, raise exception otherwise
2315     """
2316     store = X509Store()
2317     if chain:
2318         for _cert in chain:
2319             store.add_cert(_cert)
2320     store.add_cert(trust)
2322     store_ctx = _lib.X509_STORE_CTX_new()
2323     _lib.X509_STORE_CTX_init(store_ctx, store._store, cert._x509,
2325     ret = _lib.X509_verify_cert(store_ctx)
2326     if ret <= 0:
2327         _raise_current_error()

These changes were ultimately abandoned and verify_certificate() was added


"untrusted" certs

There seem to be some references to "untrusted" certificates in open issues:


And these changes seem to add support for chain certificates in the sense I
described in my original use case:


It will be good to call out the earlier changes mentioned above in these
issues so folks are warned against using the existing verify_certificate.

* Strangly, a comment in that thread actually recommends _against_ using
untrusted: "-untrusted doesn't check whether certificate chain is fully
valid. Please consider to pass both intermediate and root to command as
-CAfile as other questions suggests. – Envek Jun 22 at 18:06"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cryptography-dev/attachments/20160822/5c27b700/attachment-0001.html>

More information about the Cryptography-dev mailing list