[Twisted-Python] How to get the Python socket object for an TLS connection?
Hi, i use twisted for some volume streaming over fat pipes (10 GE) and need to tweak the rcv and send buffer sizes when streaming via socket.setsockopt(). Otherwise Twisted spends lots of time calling the producer with tiny blocksizes and writing those tiny buffers to the tiny OS buffer. Speedup for streaming was around 40x when using a large buffer instead of default block and buffer sizes, so this is really needed. I'm using Twisted 12.2. It works fine when using reactor.listenTCP(), but fails when using reactor.listenSSL() because transport.getHandle() does not return a socket object in that case (i get some SSL.Connection object instead). I already saw the TLSMemoryBIOProtocol, so there should be some way to get at the real socket object. Do i just need to skip the listenSSL() step and use the steps outlined in http://twistedmatrix.com/documents/12.2.0/api/twisted.protocols.tls.html ? import os import socket from twisted.internet import reactor from twisted.web import server, http, resource from twisted.internet.ssl import DefaultOpenSSLContextFactory port = 2000 bufsize = 4*1024*1024 class HTTPChannel(http.HTTPChannel): def connectionMade(self): sock = self.transport.getHandle() # this fails, but works when using listenTCP() sock.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, bufsize) def main(): keyfile = os.path.join('certs', 'server-key.pem') certfile = os.path.join('certs', 'server.pem') ctx = DefaultOpenSSLContextFactory(keyfile, certfile) site = server.Site(resource.NoResource()) site.protocol = HTTPChannel reactor.listenSSL(port, site, ctx) reactor.run() Michael -- Michael Schlenker Software Architect CONTACT Software GmbH Tel.: +49 (421) 20153-80 Wiener Straße 1-3 Fax: +49 (421) 20153-41 28359 Bremen http://www.contact.de/ E-Mail: msc@contact.de Sitz der Gesellschaft: Bremen Geschäftsführer: Karl Heinz Zachries, Ralf Holtgrefe Eingetragen im Handelsregister des Amtsgerichts Bremen unter HRB 13215
Hi, answering my own post at least a bit... It seems the issue is that TLSMemoryBIOProtocol shadows the getHandle() method from the original TCPTransport, if i rename it to getHandle2() in TLSMemoryBIOProtocol all works fine (but obviously other stuff will break). Looks like Ticket #5182 is only fixed 95% of the way if one needs to access getHandle() of the underlying TCPTransport. Are there any easy workarounds to get the socket handle anyway? Michael Am 12.12.2012 14:22, schrieb Michael Schlenker:
Hi,
i use twisted for some volume streaming over fat pipes (10 GE) and need to tweak the rcv and send buffer sizes when streaming via socket.setsockopt(). Otherwise Twisted spends lots of time calling the producer with tiny blocksizes and writing those tiny buffers to the tiny OS buffer. Speedup for streaming was around 40x when using a large buffer instead of default block and buffer sizes, so this is really needed.
I'm using Twisted 12.2.
It works fine when using reactor.listenTCP(), but fails when using reactor.listenSSL() because transport.getHandle() does not return a socket object in that case (i get some SSL.Connection object instead).
I already saw the TLSMemoryBIOProtocol, so there should be some way to get at the real socket object. Do i just need to skip the listenSSL() step and use the steps outlined in http://twistedmatrix.com/documents/12.2.0/api/twisted.protocols.tls.html ?
import os import socket from twisted.internet import reactor from twisted.web import server, http, resource from twisted.internet.ssl import DefaultOpenSSLContextFactory
port = 2000 bufsize = 4*1024*1024
class HTTPChannel(http.HTTPChannel): def connectionMade(self): sock = self.transport.getHandle() # this fails, but works when using listenTCP() sock.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, bufsize)
def main(): keyfile = os.path.join('certs', 'server-key.pem') certfile = os.path.join('certs', 'server.pem') ctx = DefaultOpenSSLContextFactory(keyfile, certfile) site = server.Site(resource.NoResource()) site.protocol = HTTPChannel reactor.listenSSL(port, site, ctx) reactor.run()
Michael
-- Michael Schlenker Software Architect CONTACT Software GmbH Tel.: +49 (421) 20153-80 Wiener Straße 1-3 Fax: +49 (421) 20153-41 28359 Bremen http://www.contact.de/ E-Mail: msc@contact.de Sitz der Gesellschaft: Bremen Geschäftsführer: Karl Heinz Zachries, Ralf Holtgrefe Eingetragen im Handelsregister des Amtsgerichts Bremen unter HRB 13215
On Wed, Dec 12, 2012 at 8:22 AM, Michael Schlenker
Hi,
i use twisted for some volume streaming over fat pipes (10 GE) and need to tweak the rcv and send buffer sizes when streaming via socket.setsockopt(). Otherwise Twisted spends lots of time calling the producer with tiny blocksizes and writing those tiny buffers to the tiny OS buffer. Speedup for streaming was around 40x when using a large buffer instead of default block and buffer sizes, so this is really needed.
I'm using Twisted 12.2.
It works fine when using reactor.listenTCP(), but fails when using reactor.listenSSL() because transport.getHandle() does not return a socket object in that case (i get some SSL.Connection object instead).
Probably the thing to do is listenTCP() rather than listenSSL(), and then: class YourProtocol(BaseProtocol): def connectionMade(self): self.transport.getHandle().setsockopt(...) self.transport.startTLS(contextFactory) BaseProtocol.connectionMade(self) Ideally, of course, Twisted would have an API for changing the producer/consumer buffer sizes and corresponding OS buffers. -- Itamar Turner-Trauring, Future Foundries LLC http://futurefoundries.com/ — Twisted consulting, training and support.
participants (2)
-
Itamar Turner-Trauring -
Michael Schlenker