Hello -

I'm completely new to Twisted and moderately (~1 year) experienced with Python, and I'm here because I need some help understanding how to set up a web proxy.

My goal: an HTTP proxy living on my local machine (OS X) that will transparently serve up web pages to myself, while allowing me to snoop & log connections I make to remote servers. I want the equivalent of an Attention Trust recorder (http:// www.attentiontrust.org/services) without having to use Firefox.

My first attempt: using the demonstration service.tac as a starting point, I wrote a new Resource that can view the request. I got far enough to look inside the request object passed to my resource's render() method, and return a meaningful response showing that I was seeing accurate request information. So in that render() method, I generate a new HTTPClientFactory object which *should* be able to make a request the intended server and return the results. I stopped when I realized the documentation showed examples using reactor.run (), while advising that I should never actually use reactor.run() in a real app. So I'm stuck trying to generate a new http client request while inside an http server.

My second attempt: I found the Proxy class in the twisted.web docs, which has this code snippet:

from twisted.protocols import http f = http.HTTPFactory() f.protocol = Proxy

...which is great, but I'm now I'm not sure what I need to do with the "f" object to make it work in a .tac/twistd context. It seems like should be able to add a callback in there someplace to intercept the proxied requests and pick them apart, but I'm a little stuck.

'f' is a protocol factory. It exists to create protocol instances (proxy.Proxy) when something connects to a listening socket, typically a TCP server. Here's a complete .tac file that runs a proxy on port 8080. {{{ from twisted.application import internet, service from twisted.web import http, proxy application = service.Application('proxy') f = http.HTTPFactory() f.protocol = proxy.Proxy proxyService = internet.TCPServer(8080, f) proxyService.setServiceParent(application) }}} Hope that helps. - Matt

My goal: an HTTP proxy living on my local machine (OS X) that will transparently serve up web pages to myself, while allowing me to snoop & log connections I make to remote servers. I want the equivalent of an Attention Trust recorder (http://www.attentiontrust.org/services) without having to use Firefox.

I wrote a little http proxy in twisted to help me debugging xmlrpc connections a while ago. It's just a tiny proof of concept and would require considerable improvements before getting really useful. For example it opens up a new connection for every request, let alone any error handling. Hope you'll find it useful. Johan

Hello -

I'm completely new to Twisted and moderately (~1 year) experienced with Python, and I'm here because I need some help understanding how to set up a web proxy.

My goal: an HTTP proxy living on my local machine (OS X) that will transparently serve up web pages to myself, while allowing me to snoop & log connections I make to remote servers. I want the equivalent of an Attention Trust recorder (http:// www.attentiontrust.org/services) without having to use Firefox.

My first attempt: using the demonstration service.tac as a starting point, I wrote a new Resource that can view the request. I got far enough to look inside the request object passed to my resource's render() method, and return a meaningful response showing that I was seeing accurate request information. So in that render() method, I generate a new HTTPClientFactory object which *should* be able to make a request the intended server and return the results. I stopped when I realized the documentation showed examples using reactor.run (), while advising that I should never actually use reactor.run() in a real app. So I'm stuck trying to generate a new http client request while inside an http server.

My second attempt: I found the Proxy class in the twisted.web docs, which has this code snippet:

from twisted.protocols import http f = http.HTTPFactory() f.protocol = Proxy

...which is great, but I'm now I'm not sure what I need to do with the "f" object to make it work in a .tac/twistd context. It seems like should be able to add a callback in there someplace to intercept the proxied requests and pick them apart, but I'm a little stuck.

'f' is a protocol factory. It exists to create protocol instances (proxy.Proxy) when something connects to a listening socket, typically a TCP server. Here's a complete .tac file that runs a proxy on port 8080. {{{ from twisted.application import internet, service from twisted.web import http, proxy application = service.Application('proxy') f = http.HTTPFactory() f.protocol = proxy.Proxy proxyService = internet.TCPServer(8080, f) proxyService.setServiceParent(application) }}} Hope that helps. - Matt

My goal: an HTTP proxy living on my local machine (OS X) that will transparently serve up web pages to myself, while allowing me to snoop & log connections I make to remote servers. I want the equivalent of an Attention Trust recorder (http://www.attentiontrust.org/services) without having to use Firefox.

I wrote a little http proxy in twisted to help me debugging xmlrpc connections a while ago. It's just a tiny proof of concept and would require considerable improvements before getting really useful. For example it opens up a new connection for every request, let alone any error handling. Hope you'll find it useful. Johan