[Twisted-Python] SSL: Getting the client certificate
Hi, following Eli Criffields nice example [1] I implemented a small SSL server with Twisted. My server should not only verify the client certificate, but also check the Common Name (CN) against a whitelist. All this should happen before any user data is exchanged. Verifying the client certificate worked nicely, but I couldn't access its contents: transport.getPeerCertificate() always returned 'None'. Apparently Eli had the same problem [2]. After some testing with PyOpenSSL now I think I have found a solution: Before we can get the client certificate, we have to make sure that the SSL handshake has taken place. (If it hasn't, there simply is no client certificate to deal with yet.) This can be done by calling the do_handshake() method of the underlying socket. The SSL handshake takes some time so we will have to try several times. Here's an (incomplete) example showing the interesting part: - - cut --- import OpenSSL class MyProtocol(Protocol): def connectionMade(self): # Make sure that SSL handshake has taken place while True: try: self.transport.socket.do_handshake() break except OpenSSL.SSL.WantReadError: pass clientCert = self.transport.getPeerCertificate() if clientCert is None: log.msg("No client cert available.") else: subject = clientCert.get_subject() log.msg("Subject: %s" % subject) log.msg("Common Name: %s" % subject.CN) - - cut --- If you see a nicer way to wait for the SSL handshake please let me know. Using time.sleep() didn't work for me. Side note: Getting the certificate in a dataReceived() instead of connectionMade() works without manually doing the handshake. I think this is because the underlying PyOpenSSL recv() method handles the handshake for us. But at least for my purpose it makes more sense to verify the client cert right upon connection, before any user data is exchanged. Regards, Dirk [1] http://archives.free.net.ph/message/20070511.203607.36001e38.en.html [2] http://archives.free.net.ph/message/20070607.211438.9354342f.en.html
On Sun, 26 Aug 2007 20:26:44 +0200, Dirk Loss <lists@dirk-loss.de> wrote:
Hi,
following Eli Criffields nice example [1] I implemented a small SSL server with Twisted. My server should not only verify the client certificate, but also check the Common Name (CN) against a whitelist. All this should happen before any user data is exchanged.
Verifying the client certificate worked nicely, but I couldn't access its contents: transport.getPeerCertificate() always returned 'None'. Apparently Eli had the same problem [2].
After some testing with PyOpenSSL now I think I have found a solution:
Before we can get the client certificate, we have to make sure that the SSL handshake has taken place. (If it hasn't, there simply is no client certificate to deal with yet.) This can be done by calling the do_handshake() method of the underlying socket. The SSL handshake takes some time so we will have to try several times.
This is basically a bug in Twisted's SSL support. I forget if there's a way to fix it using PyOpenSSL or if it's a limitation of the bindings.
Here's an (incomplete) example showing the interesting part: - - cut --- import OpenSSL
class MyProtocol(Protocol):
def connectionMade(self):
# Make sure that SSL handshake has taken place while True: try: self.transport.socket.do_handshake() break except OpenSSL.SSL.WantReadError: pass
clientCert = self.transport.getPeerCertificate() if clientCert is None: log.msg("No client cert available.") else: subject = clientCert.get_subject() log.msg("Subject: %s" % subject) log.msg("Common Name: %s" % subject.CN) - - cut ---
If you see a nicer way to wait for the SSL handshake please let me know. Using time.sleep() didn't work for me.
This solution has at least two related problems: * it will block the reactor until the handshake for that client completes, which means no other I/O will occur and none other application code will be able to run. This might be fine for your application, but in general it's not a very good thing. * if a malicious client connects, they can just never complete the handshake and your server will hang in that loop indefinitely.
Side note: Getting the certificate in a dataReceived() instead of connectionMade() works without manually doing the handshake. I think this is because the underlying PyOpenSSL recv() method handles the handshake for us. But at least for my purpose it makes more sense to verify the client cert right upon connection, before any user data is exchanged.
The ideal solution would be to fix the bug in Twisted's SSL support so that connectionMade is called at the right time. Another possible solution might be to do your verification using the SSL context object. CertificateOptions might give you some ideas about how to do this: http://twistedmatrix.com/documents/current/api/twisted.internet.ssl.Certific... Jean-Paul
Jean-Paul Calderone wrote:
This solution has at least two related problems: * it will block the reactor until the handshake for that client completes,
Yes, it's ugly. Not being able to run other application code is no problem in my case, so I thought I could live with that. But the possibe DoS attack you mentioned should be avoided, of course.
The ideal solution would be to fix the bug in Twisted's SSL support so that connectionMade is called at the right time.
This would be nice. Maybe I should take a deeper look at Twisted's SSL code.
Another possible solution might be to do your verification using the SSL context object.
Could you elaborate on this? I think I am already using the SSL context object to do the verification: theCert = ssl.PrivateCertificate.loadPEM(open(myKey).read()) theCA = ssl.Certificate.loadPEM(open(trustedCA).read()) ctx = theCert.options(theCA) ctx.verify = True ctx.verifyDepth = 9 ctx.requireCertificate = True ctx.verifyOnce = False application = service.Application("MySSLServer") MyService = internet.SSLServer(listenport, MyAppFactory(), ctx) MyService.setServiceParent(application) (Above code obviously will not run. Just to show the basic steps I take.) Regards Dirk
On Sun, 26 Aug 2007 22:23:49 +0200, Dirk Loss <lists@dirk-loss.de> wrote:
Jean-Paul Calderone wrote:
This solution has at least two related problems: * it will block the reactor until the handshake for that client completes,
Yes, it's ugly. Not being able to run other application code is no problem in my case, so I thought I could live with that. But the possibe DoS attack you mentioned should be avoided, of course.
The ideal solution would be to fix the bug in Twisted's SSL support so that connectionMade is called at the right time.
This would be nice. Maybe I should take a deeper look at Twisted's SSL code.
Another possible solution might be to do your verification using the SSL context object.
Could you elaborate on this? I think I am already using the SSL context object to do the verification:
theCert = ssl.PrivateCertificate.loadPEM(open(myKey).read()) theCA = ssl.Certificate.loadPEM(open(trustedCA).read()) ctx = theCert.options(theCA) ctx.verify = True ctx.verifyDepth = 9 ctx.requireCertificate = True ctx.verifyOnce = False application = service.Application("MySSLServer") MyService = internet.SSLServer(listenport, MyAppFactory(), ctx) MyService.setServiceParent(application)
(Above code obviously will not run. Just to show the basic steps I take.)
CertificateOptions doesn't directly support a custom verification callback, which I think is what you want, but it does use that feature in order to log SSL errors. If you take a look at the implementation, you'll see a nested function named _trackVerificationProblems. If this could be customized by application code, then it could do things like check hostnames. I forget why this isn't exposed to applications. I think someone suggested that it wasn't actually what you wanted to do, but I don't remember any more details than that. Jean-Paul
Dirk Loss <lists@dirk-loss.de> writes:
Jean-Paul Calderone wrote: (...)
Another possible solution might be to do your verification using the SSL context object.
Could you elaborate on this? I think I am already using the SSL context object to do the verification:
Not sure if it helps, but here's some old code of mine where I experimented with the echo SSL examples to add symmetric certificate checking. Just checked and it seems ok with Python 2.5.1 and Twisted 2.5.0 (pyOpenSSL 0.6). It uses direct SSL context objects rather than the Twisted wrapper versions. To be honest, at the time it was because I was still feeling my way around the SSL support and found using the direct context easier, but I believe you do have full access to the certificate in the context's _verify method. Returning 0/False from _verify rather than just propagating ok can reject the handshake. (Note that _verify can be called multiple times during the sequence as well as in cases where ok is already 0 I believe). There are a bunch of debugging prints still in the code where I was seeing what sort of stuff was available to the context factory/verification. -- David echoserv_ssl.py: --------------- # Twisted, the Framework of Your Internet # Copyright (C) 2001 Matthew W. Lefkowitz # # This library is free software; you can redistribute it and/or # modify it under the terms of version 2.1 of the GNU Lesser General Public # License as published by the Free Software Foundation. # # This library is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU # Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public # License along with this library; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA from OpenSSL import SSL, crypto class ServerContextFactory: def _verify(self, connection, x509, errnum, errdepth, ok): print '_verify (ok=%d):' % ok print ' subject:', x509.get_subject() print ' issuer:', x509.get_issuer() print ' errnum %s, errdepth %d' % (errnum, errdepth) return False # ok def getContext(self): """Create an SSL context. This is a sample implementation that loads a certificate from a file called 'server.pem'.""" ctx = SSL.Context(SSL.SSLv23_METHOD) ctx.use_certificate_file('server.pem') ctx.use_privatekey_file('server.pem') print 'Context additions' ctx.load_client_ca('ca/all-cas.cert') ctx.load_verify_locations('ca/ca.cert') ctx.set_verify(SSL.VERIFY_PEER|SSL.VERIFY_FAIL_IF_NO_PEER_CERT, self._verify) print 'verify depth:', ctx.get_verify_depth() ctx.set_verify_depth(10) print 'verify depth:', ctx.get_verify_depth() return ctx import echoserv class MyProtocol(echoserv.Echo): def connectionMade(self): print 'connectionMade', self.transport.getPeerCertificate() return echoserv.Echo.connectionMade(self) def dataReceived(self, data): print 'dataReceived', self.transport.getPeerCertificate() return echoserv.Echo.dataReceived(self, data) if __name__ == '__main__': import echoserv, sys from twisted.internet.protocol import Factory from twisted.internet import ssl, reactor from twisted.python import log log.startLogging(sys.stdout) factory = Factory() factory.protocol = MyProtocol reactor.listenSSL(9000, factory, ServerContextFactory()) reactor.run() echoclient_ssl.py: ----------------- # Twisted, the Framework of Your Internet # Copyright (C) 2001 Matthew W. Lefkowitz # # This library is free software; you can redistribute it and/or # modify it under the terms of version 2.1 of the GNU Lesser General Public # License as published by the Free Software Foundation. # # This library is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU # Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public # License along with this library; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA from OpenSSL import SSL import sys from twisted.internet.protocol import ClientFactory from twisted.protocols.basic import LineReceiver from twisted.internet import ssl, reactor import inspect class ClientContextFactory(ssl.ClientContextFactory): def _verify(self, connection, x509, errnum, errdepth, ok): print '_verify (ok=%d):' % ok print ' subject:', x509.get_subject() print ' issuer:', x509.get_issuer() print ' errnum %s, errdepth %d' % (errnum, errdepth) return ok def getContext(self): ctx = ssl.ClientContextFactory.getContext(self) ctx.use_certificate_file('client.pem') ctx.use_privatekey_file('client.pem') ctx.load_verify_locations('ca/ca.cert') ctx.set_verify(SSL.VERIFY_PEER|SSL.VERIFY_FAIL_IF_NO_PEER_CERT, self._verify) return ctx class EchoClient(LineReceiver): end="Bye-bye!" def connectionMade(self): self.sendLine("Hello, world!") self.sendLine("What a fine day it is.") self.sendLine(self.end) def connectionLost(self, reason): print 'connection lost (protocol)' def lineReceived(self, line): x509 = self.transport.getPeerCertificate() methods = [x for x in dir(x509) if callable(getattr(x509,x)) and not (x.startswith('set_') or x.startswith('add_') or x.startswith('gmtime_') or x in ('sign','digest'))] for m in methods: print m, getattr(x509,m)() print "receive:", line if line==self.end: self.transport.loseConnection() class EchoClientFactory(ClientFactory): protocol = EchoClient def clientConnectionFailed(self, connector, reason): print 'connection failed:', reason.getErrorMessage() reactor.stop() def clientConnectionLost(self, connector, reason): print 'connection lost:', reason.getErrorMessage() reactor.stop() def main(): if len(sys.argv) > 1: host = sys.argv[1] else: host = 'localhost' factory = EchoClientFactory() reactor.connectSSL(host, 9000, factory, ClientContextFactory()) reactor.run() if __name__ == '__main__': main()
David Bolen wrote:
Not sure if it helps, but here's some old code of mine where I experimented with the echo SSL examples to add symmetric certificate checking.
Thank you very much for your code. It's useful to see that the peer cert could be interpreted in the _verify callback. And your method to retrieve all X.509 attributes is quite clever. :) But I could not get your SSL server and client to work. At least not completely: After printing the server's certificates (server.pem and ca.pem), the client quits with the following error: _verify (ok=1): ... errnum 0, errdepth 0 connection lost (protocol) connection lost: [('SSL routines', 'SSL3_READ_BYTES', 'sslv3 alert certificate unknown'), ('SSL routines', 'SSL3_READ_BYTES', 'ssl handshake failure')] Maybe there's some problem with my certificates. I had successfully tested them with OpenSSL, though: openssl s_server -accept 9000 -Verify 9 -cert server.pem -CAfile ca.pem openssl s_client -connect localhost:9000 -verify 9 -cert client.pem -CAfile ca.pem (In your code I replaced the "ca/ca.cert" and "ca/all-cas.cert" filenames with "ca.pem".) I get the same error if I use your server and OpenSSL as a client. If you have any idea, I'll be happy to hear from you again. For now I will just use use the second method I mentioned in my original post: getting the peer certificate in dataReceived() instead of connectionMade(). This at least avoids the ugly do_handshake() polling loop I had been using. But there's still some DoS risk because a malicious client could just wait forever before sending any data (and thus before authentication.) Regards Dirk
Dirk Loss <lists@dirk-loss.de> writes:
But I could not get your SSL server and client to work. At least not completely: After printing the server's certificates (server.pem and ca.pem), the client quits with the following error:
Did you fix the stray "False" I left in the server _verify routine? I just noticed that it was still there from when I tested (before writing the response), so instead of returning the value of "ok" it's explicitly failing in all cases. Sorry about that. -- David
participants (3)
-
David Bolen -
Dirk Loss -
Jean-Paul Calderone