[Twisted-Python] Implementing STARTTLS in a protocol

Howdy list, I'm trying to implement a protocol using Twisted which has a "STARTTLS" command to switch the protocol from plain TCP to TCP over TLS. I've mostly been going by the way that the imap4.py module seems to do it, but I can't seem to get a handshake to complete. I found this page ( http://wiki.vislab.usyd.edu.au/moin.cgi/SSLCertNotes ) which was helpful, but I don't want to force client cert authentication. In order to separate this problem from other issues, I've adapted the echo protocol code from above the above page to try and get a simple test case (my code below) I am recieving the following output and traceback when running the client code ( on both Windows and Linux ): using TLSv1:
tls_echoclient.py
Sending: Hello, world! receive: ERROR: Must authenticate Sending: STARTTLS receive: READY Sending: Continuing connection lost (protocol) connection lost: [('SSL routines', 'SSL3_READ_BYTES', 'sslv3 alert handshake failure'), ('SSL routines', 'SSL3_READ_BYTES', 'ssl handshake failure')] Traceback (most recent call last): File "C:\Documents and Settings\kevinh\Desktop\mine_id\sandbox\funsize\sslecho\tls_echoclient.py", line 58, in <module> reactor.run() File "C:\Python25\lib\site-packages\twisted\internet\posixbase.py", line 223, in run self.mainLoop() File "C:\Python25\lib\site-packages\twisted\internet\posixbase.py", line 234, in mainLoop self.doIteration(t) File "C:\Python25\lib\site-packages\twisted\internet\selectreactor.py", line 140, in doSelect _logrun(selectable, _drdw, selectable, method, dict) --- <exception caught here> --- File "C:\Python25\lib\site-packages\twisted\python\log.py", line 51, in callWithLogger return callWithContext({"system": lp}, func, *args, **kw) << SNIP >> File "C:\Python25\lib\site-packages\twisted\internet\base.py", line 490, in stop "Can't stop reactor that isn't running.") twisted.internet.error.ReactorNotRunning: Can't stop reactor that isn't running. What am I doing wrong? Is there a SSL config option I'm setting incorrectly? Do I need to use a different SSL Context? Am I totally off base? Thanks, Kevin Horn ========= tls_echoserver.py # adapted from: http://wiki.vislab.usyd.edu.au/moin.cgi/SSLCertNotes import sys from OpenSSL import SSL from twisted.python import log from twisted.internet import ssl, reactor from twisted.protocols.basic import LineReceiver from twisted.internet.protocol import Factory private_key_file = "privkey_dsa.pem" cert_file_name = "cacert.pem" sslmethod = ssl.SSL.TLSv1_METHOD #~ sslmethod = ssl.SSL.SSLv23_METHOD class Echo(LineReceiver): donetls = 0 def sendLine(self, line): print "Sending:",line LineReceiver.sendLine(self,line) def lineReceived(self, line): print "Got:",line if self.donetls: print "Sending OK" self.transport.write("OK") elif line == "STARTTLS": self.sendLine("READY") #~ self.tlsCtx = CtxFactory() self.tlsCtx = ssl.DefaultOpenSSLContextFactory(private_key_file, cert_file_name, sslmethod) self.transport.startTLS(self.tlsCtx) self.donetls = 1 else: self.sendLine("ERROR: Must authenticate") factory = Factory() factory.protocol = Echo reactor.listenTCP(8000, factory) reactor.run() ========= tls_echoclient.py # adapted from: http://wiki.vislab.usyd.edu.au/moin.cgi/SSLCertNotes from OpenSSL import SSL import sys from twisted.internet.protocol import ClientFactory from twisted.protocols.basic import LineReceiver from twisted.internet import ssl, reactor class EchoClient(LineReceiver): end="Bye-bye!" def connectionMade(self): self.sendLine("Hello, world!") # Signals error def connectionLost(self, reason): print 'connection lost (protocol)' reactor.stop() def sendLine(self, line): print "Sending:",line LineReceiver.sendLine(self,line) def lineReceived(self, line): print "receive:", line if line == "ERROR: Must authenticate": self.sendLine("STARTTLS") elif line == "READY": #~ self.ctx = CxtFactory() self.ctx = ssl.ClientContextFactory() self.ctx.method = ssl.SSL.TLSv1_METHOD #~ self.ctx.method = ssl.SSL.SSLv23_METHOD self.transport.startTLS(self.ctx) self.sendLine("Continuing"); else: self.sendLine("wibble") class EchoClientFactory(ClientFactory): protocol = EchoClient def clientConnectionFailed(self, connector, reason): print 'connection failed:', reason.getErrorMessage() reactor.stop() def clientConnectionLost(self, connector, reason): print 'connection lost:', reason.getErrorMessage() reactor.stop() factory = EchoClientFactory() reactor.connectTCP('localhost', 8000, factory) reactor.run()

On Fri, 23 May 2008 12:29:44 -0500, Kevin Horn <kevin.horn@gmail.com> wrote:
The traceback here is just because you're calling reactor.stop() twice, once in Protocol.connectionLost, then again in Factory.clientConnectionLost. Get rid of one of these and at least you'll get rid of some spurious noise. As far as the TLS part of your code goes, it basically looks okay. By doing a sendLine immediately before you call startTLS, you risk running into #686, but if you actually hit that, you should see a warning and the connection should be closed without an OpenSSL error. So I'm not exactly sure what problem you're encountering. To further complicate matters, when I run your code, TLS is successfully negotiated. Jean-Paul

On Fri, May 23, 2008 at 3:32 PM, Jean-Paul Calderone <exarkun@divmod.com> wrote:
Thanks for responding, Jean-Paul, and thanks for the tip. I've been so consumed with reading through the noise that for some reason it never occurred to me to try and get rid of it. As far as the TLS part of your code goes, it basically looks okay. By doing
Jean-Paul Well that's ... frustrating. I was hoping I had just overlooked something obvious (and easy to fix!) Can you tell me more about the environment you are running under? So far I've tried: WinXP, Python 2.5, Twisted 8.0.1, pyOpenSSL 0.7, OpenSSL 0.9.8g Linux(CentOS), Python 2.4, Twisted 8.1.0, pyOpenSSL 0.7, OpenSSL 0.9.7a Perhaps there is something wrong with my certificates? I would expect that this would cause errors on the server end, though... Is there any way to get more information about the handshake failure? Thanks, Kevin Horn

On Fri, 23 May 2008 17:31:39 -0500, Kevin Horn <kevin.horn@gmail.com> wrote:
I tried with Ubuntu 7.10, Python 2.5.1, Twisted trunk@HEAD, OpenSSL 0.9.8e-5ubuntu3, pyOpenSSL 0.6-2.3ubuntu1. I also tried with the Twisted 2.5 release branch. I don't have pyOpenSSL 0.7 handy at the moment. Maybe you could give 0.6 a try? I can't think of any changes between those versions that might be causing this, but one never knows. Jean-Paul
Perhaps there is something wrong with my certificates? I would expect that this would cause errors on the server end, though...
Seems like the cert is probably fine, yea. I'm attaching the one I tested with so you can give it a try, though.
Is there any way to get more information about the handshake failure?
ssldump might tell you something, but you'll have to dig a bit and do some interpretation. And it might just end up telling you that the handshake is failing. Jean-Paul

On Sat, May 24, 2008 at 10:57 AM, Jean-Paul Calderone <exarkun@divmod.com> wrote:
OK, tested on Win2K, Python 2.4, Twisted 8.1.0, PyOpenSSL 0.7, OpenSSL 0.9.8g using your cert...TLS handshake succeeded. using my cert...same error as before so there definitely seems to be something wrong with my certificate files What procedure and/or OpenSSL commands did you use to create yours? Maybe I'm doing something incorrectly when creating certs? Nice to see my code working though...thanks for your help so far! Kevin Horn

On Fri, 23 May 2008 12:29:44 -0500, Kevin Horn <kevin.horn@gmail.com> wrote:
The traceback here is just because you're calling reactor.stop() twice, once in Protocol.connectionLost, then again in Factory.clientConnectionLost. Get rid of one of these and at least you'll get rid of some spurious noise. As far as the TLS part of your code goes, it basically looks okay. By doing a sendLine immediately before you call startTLS, you risk running into #686, but if you actually hit that, you should see a warning and the connection should be closed without an OpenSSL error. So I'm not exactly sure what problem you're encountering. To further complicate matters, when I run your code, TLS is successfully negotiated. Jean-Paul

On Fri, May 23, 2008 at 3:32 PM, Jean-Paul Calderone <exarkun@divmod.com> wrote:
Thanks for responding, Jean-Paul, and thanks for the tip. I've been so consumed with reading through the noise that for some reason it never occurred to me to try and get rid of it. As far as the TLS part of your code goes, it basically looks okay. By doing
Jean-Paul Well that's ... frustrating. I was hoping I had just overlooked something obvious (and easy to fix!) Can you tell me more about the environment you are running under? So far I've tried: WinXP, Python 2.5, Twisted 8.0.1, pyOpenSSL 0.7, OpenSSL 0.9.8g Linux(CentOS), Python 2.4, Twisted 8.1.0, pyOpenSSL 0.7, OpenSSL 0.9.7a Perhaps there is something wrong with my certificates? I would expect that this would cause errors on the server end, though... Is there any way to get more information about the handshake failure? Thanks, Kevin Horn

On Fri, 23 May 2008 17:31:39 -0500, Kevin Horn <kevin.horn@gmail.com> wrote:
I tried with Ubuntu 7.10, Python 2.5.1, Twisted trunk@HEAD, OpenSSL 0.9.8e-5ubuntu3, pyOpenSSL 0.6-2.3ubuntu1. I also tried with the Twisted 2.5 release branch. I don't have pyOpenSSL 0.7 handy at the moment. Maybe you could give 0.6 a try? I can't think of any changes between those versions that might be causing this, but one never knows. Jean-Paul
Perhaps there is something wrong with my certificates? I would expect that this would cause errors on the server end, though...
Seems like the cert is probably fine, yea. I'm attaching the one I tested with so you can give it a try, though.
Is there any way to get more information about the handshake failure?
ssldump might tell you something, but you'll have to dig a bit and do some interpretation. And it might just end up telling you that the handshake is failing. Jean-Paul

On Sat, May 24, 2008 at 10:57 AM, Jean-Paul Calderone <exarkun@divmod.com> wrote:
OK, tested on Win2K, Python 2.4, Twisted 8.1.0, PyOpenSSL 0.7, OpenSSL 0.9.8g using your cert...TLS handshake succeeded. using my cert...same error as before so there definitely seems to be something wrong with my certificate files What procedure and/or OpenSSL commands did you use to create yours? Maybe I'm doing something incorrectly when creating certs? Nice to see my code working though...thanks for your help so far! Kevin Horn
participants (2)
-
Jean-Paul Calderone
-
Kevin Horn