[Twisted-Python] RE: Modifying a web proxy to use SSL
![](https://secure.gravatar.com/avatar/b36a755da3ce192fe47153afe99e99b4.jpg?s=120&d=mm&r=g)
Here is a little more detail on my problem. I've been able to verify that the connection is made to the https server and that data is received. In fact, I've overridden the ProxyRequest.handleResponseEnd() method to save this data to a database. I can see the raw html and image data in the database, but nothing is returned to the browser. Does anyone have any experience implementing their own proxy with SSL that might be able to point me in the right direction? Any assistance would be greatly appreciated. -larry Here is my custom ProxyRequest class (i had to modify some stuff because secure urls have the https stripped and :443 added to the uri for some reason): class SerpicoProxyRequest(proxy.ProxyRequest): #protocols = {'http': SerpicoProxyClientFactory} #LEA - added https protocols = {'http': SerpicoProxyClientFactory, 'https' : SerpicoProxyClientFactory} ports = {'http' : 80, 'https' : 443} def __init__(self, *args): proxy.ProxyRequest.__init__(self, *args) def process(self): if (self.uri.find(':443')>0): self.uri = 'https://' + self.uri.replace(':443','') self.method = 'GET' parsed = urlparse.urlparse(self.uri) protocol = parsed[0] host = parsed[1] port = self.ports[protocol] self.setHost(host,port,1) if ':' in host: host, port = host.split(':') port = int(port) rest = urlparse.urlunparse(('','')+parsed[2:]) if not rest: rest = rest+'/' class_ = self.protocols[protocol] headers = self.getAllHeaders().copy() if not headers.has_key('host'): headers['host'] = host self.content.seek(0,0) s = self.content.read() clientFactory = class_(self.method, rest, self.clientproto, headers, s, self) clientContextFactory = ssl.ClientContextFactory() c = reactor.connectSSL(host, port, clientFactory, clientContextFactory) else: proxy.ProxyRequest.process(self) Here is my custom ProxyClient (the Page() object is what I'm using to save the data to the database): class SerpicoProxyClient(proxy.ProxyClient): def connectionMade(self): global CURRENTJOBID global CURRENTCHILDID log.msg("initializing connection") self.page = Page() if not self.father.uri == "%s/command/get_agent_task/%s" % (command_site,machine_name): self.page.instance = machine_name self.page.job_id = int(CURRENTJOBID or 0) self.page.task_id = int(CURRENTCHILDID or 0) # self.father refers to the http.Request object self.page.request_uri = self.father.uri self.page.save() log.msg("MACHINE NAME IS %s" % machine_name) log.msg("PROXYREQUEST %s" % self.father.uri) log.msg("TRANSPORT %s" % self.transport) log.msg("ISSECURE %s" % self.father.isSecure()) proxy.ProxyClient.connectionMade(self) def handleHeader(self, key, value): proxy.ProxyClient.handleHeader(self, key, value) # log.msg("%s : %s added" % (key, value)) if not self.father.uri == "%s/command/get_agent_task/%s" % (command_site,machine_name): self.page.header_set.create(key=key, value=value) def handleResponsePart(self, data): if not self.father.uri == "%s/command/get_agent_task/%s" % (command_site,machine_name): self.page.data = data proxy.ProxyClient.handleResponsePart(self, data) def handleResponseEnd(self): if not self.father.uri == "%s/command/get_agent_task/%s" % (command_site,machine_name): self.page.save() log.msg('ending response with my data::%s\n\nfatherdata::%s\n\nmytransport::%s' % (self.page.data, self.father.c hannel.transport, self.transport)) self.transport.loseConnection() self.father.channel.transport.loseConnection() ________________________________ From: Adams, Larry Sent: Tuesday, May 29, 2007 3:42 PM To: twisted-python@twistedmatrix.com Subject: Modifying a web proxy to use SSL I'm trying to use twisted to proxy all web requests for a set of PCs (virtual machines actually). I've created my own ProxyRequest class to override the process() method so that it will use SSL if the URL is secure. It makes the connection and retrieves the data successfully, but the browser still doesn't display the HTML. I must be missing some other class/method that I have to override, but I'm at a loss as to which one at the moment. Below is some logging from my application for the same page called via http and https respectively. Any help or pointers to working examples would be greatly appreciated! Thanks, Larry Adams 2007/05/29 15:10 -0500 [SerpicoProxy,11,172.18.36.22] Starting factory <__main__.SerpicoProxyClientFactory instance at 0xb71d8cec> 2007/05/29 15:10 -0500 [Uninitialized] building protocol for addr: IPv4Address(TCP, 'www.connectcommerce.com <file://www.connectcommerce.com> ', 80) 2007/05/29 15:10 -0500 [Uninitialized] initializing connection 2007/05/29 15:10 -0500 [Uninitialized] MACHINE NAME IS surfsidekick 2007/05/29 15:10 -0500 [Uninitialized] PROXYREQUEST http://www.connectcommerce.com/ <http://www.connectcommerce.com/> 2007/05/29 15:10 -0500 [Uninitialized] TRANSPORT <<class 'twisted.internet.tcp.Client'> to ('www.connectcommerce.com <file://www.connectcommerce.com> ', 80) at b71d8ccc> 2007/05/29 15:10 -0500 [Uninitialized] ISSECURE False 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] Date : Tue, 29 May 2007 20:11:27 GMT added 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] Server : Apache/1.3.33 (Unix) (Gentoo/Linux) added 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] Location : http://www.connectcommerce.com/global/login.html <http://www.connectcommerce.com/global/login.html> added 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] Connection : close added 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] Content-Type : text/html; charset=iso-8859-1 added 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] trying to lose connection for transport: <<class 'twisted.internet.tcp.Client'> to ('www.connectcommerce.com <file://www.connectcommerce.com> ', 80) at b71d8ccc> 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] parent transport : <SerpicoProxy #11 on 74007> 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] Stopping factory <__main__.SerpicoProxyClientFactory instance at 0xb71d8cec> 2007/05/29 15:11 -0500 [SerpicoProxy,18,172.18.36.22] Starting factory <__main__.SerpicoProxyClientFactory instance at 0xb71d8d0c> 2007/05/29 15:11 -0500 [Uninitialized] building protocol for addr: IPv4Address(TCP, 'www.connectcommerce.com <file://www.connectcommerce.com> ', 443) 2007/05/29 15:11 -0500 [Uninitialized] initializing connection 2007/05/29 15:11 -0500 [Uninitialized] MACHINE NAME IS surfsidekick 2007/05/29 15:11 -0500 [Uninitialized] PROXYREQUEST https://www.connectcommerce.com <https://www.connectcommerce.com> 2007/05/29 15:11 -0500 [Uninitialized] TRANSPORT <<class 'twisted.internet.tcp.TLSConnection'> to ('www.connectcommerce.com <file://www.connectcommerce.com> ', 443) at b71bd86c> 2007/05/29 15:11 -0500 [Uninitialized] ISSECURE True 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] Date : Tue, 29 May 2007 20:12:00 GMT added 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] Server : Apache/1.3.33 (Unix) (Gentoo/Linux) added 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] Location : http://www.connectcommerce.com/global/login.html <http://www.connectcommerce.com/global/login.html> added 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] Connection : close added 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] Content-Type : text/html; charset=iso-8859-1 added 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] trying to lose connection for transport: <<class 'twisted.internet.tcp.TLSConnection'> to ('www.connectcommerce.com <file://www.connectcommerce.com> ', 443) at b71bd86c> 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] parent transport : <SerpicoProxy #18 on 74007> 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] Stopping factory <__main__.SerpicoProxyClientFactory instance at 0xb71d8d0c>
participants (1)
-
Adams, Larry