[Twisted-Python] HTTPS proxy with twisted?
Hi, I am doing a research project for a security class that involves writing an HTTP and HTTPS proxy. The intent of this proxy is to ensure that communications that should be secure are secure, without bombarding the user with false warnings about potential man-in-the-middle attacks, for example, when a website legitimately uses a self-signed certificate. I will need to write an HTTPS proxy, which will examine the certificates sent from the web server and determine whether the certificate is valid or invalid. If the proxy determines if the certificate is valid, I will need to resign the document. I suppose this will require that the proxy be a certificate authority and will generate certificates for websites, which the proxy will then use to sign the documents. Will this be something that twisted can do easily? If so, where can I find documentation for how to do this? Thanks, Steve
On 11/03/12 16:18, Steve Chapel wrote:
I will need to write an HTTPS proxy, which will examine the certificates sent from the web server and determine whether the certificate is valid or invalid. If the proxy determines if the certificate is valid, I will need to resign the document. I suppose this will require that the proxy be a certificate authority and will generate certificates for websites, which the proxy will then use to sign the documents. Will this be something that twisted can do easily? If so, where can I find documentation for how to do this?
This is a pretty hard question to answer in this form, and depends on what you mean by "easily". Since you say it's classwork I'm reluctant to say too much, but... Fundamentally, the only "difficult" bit of this project in terms of Twisted capabilities is finding the original destination address of your intercepted connections (so that you can do a "lookaside" connection and verify / impersonate the far-end cert) Presumably you'll be using something like Linux/IPTables to do this: iptables t nat -A PREROUTING \ -p tcp --dport 443 -j REDIRECT --to-port <twisted> In that case, you can find the original destination address by calling: socket.getsockopt(self.transport.fileno(), SOL_IP, SO_ORIGINAL_DST, 16) ...in your transport "connectionMade". You will presumably then want to start up an SSL connection to the original IP (or draw from cache) to find the far-end cert attributes (note: plural), call out to your local MITM CA for an impersonated cert/key, then call startTLS in server mode using a context holding the fake cert/key. This isn't very hard, and Twisted has everything you need (accept TCP connections, make outgoing SSL, find server certs, call out to subprocess, startTLS in server mode) except the SO_ORIGINAL_DST stuff (which is easy to add in). Anyway, I hope this helps; good luck with the assignment! Cheers, Phil
Well, the HTTPS proxy actually not an assignment. Getting the HTTPS proxy working is just the starting point we need before we can do any research, and at this point the research project is due in four weeks. If you can point to some twisted code for an HTTPS proxy or documentation for how to write one, that won't be giving us undue help but will enable us to have a chance to get the research done at all. At this point, I'm thinking of using a regular Python program because I found a MITM proxy already written, so we could start the research right away. -- Steve On 03/12/2012 11:33 AM, Phil Mayers wrote:
On 11/03/12 16:18, Steve Chapel wrote:
I will need to write an HTTPS proxy, which will examine the certificates sent from the web server and determine whether the certificate is valid or invalid. If the proxy determines if the certificate is valid, I will need to resign the document. I suppose this will require that the proxy be a certificate authority and will generate certificates for websites, which the proxy will then use to sign the documents. Will this be something that twisted can do easily? If so, where can I find documentation for how to do this? This is a pretty hard question to answer in this form, and depends on what you mean by "easily". Since you say it's classwork I'm reluctant to say too much, but...
Fundamentally, the only "difficult" bit of this project in terms of Twisted capabilities is finding the original destination address of your intercepted connections (so that you can do a "lookaside" connection and verify / impersonate the far-end cert)
Presumably you'll be using something like Linux/IPTables to do this:
iptables t nat -A PREROUTING \ -p tcp --dport 443 -j REDIRECT --to-port<twisted>
In that case, you can find the original destination address by calling:
socket.getsockopt(self.transport.fileno(), SOL_IP, SO_ORIGINAL_DST, 16)
...in your transport "connectionMade". You will presumably then want to start up an SSL connection to the original IP (or draw from cache) to find the far-end cert attributes (note: plural), call out to your local MITM CA for an impersonated cert/key, then call startTLS in server mode using a context holding the fake cert/key.
This isn't very hard, and Twisted has everything you need (accept TCP connections, make outgoing SSL, find server certs, call out to subprocess, startTLS in server mode) except the SO_ORIGINAL_DST stuff (which is easy to add in).
Anyway, I hope this helps; good luck with the assignment!
Cheers, Phil
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On 12/03/12 16:11, Steve Chapel wrote:
Well, the HTTPS proxy actually not an assignment. Getting the HTTPS proxy working is just the starting point we need before we can do any research, and at this point the research project is due in four weeks. If you can point to some twisted code for an HTTPS proxy or documentation for how to write one, that won't be giving us undue help but will enable us to have a chance to get the research done at all. At this point, I'm thinking of using a regular Python program because I found a MITM proxy already written, so we could start the research right away.
I got curious about how difficult this would be, and spent a bit of time on it last night. See here: https://github.com/philmayers/txsslmitm This is MIT licensed, as per Twisted. It is really a proof-of-concept, but it does work in my tests. I'm impressed how few lines of code it takes in Twisted to do this; it really is an awesome framework!
Hi I wrote a proxy designed to lie once. It's called minitrue. SSL MITMing is still a feature I hope that it will grow one day (see issue tracker). It'll probably be a lot easier to hack this on top of minitrue than starting from scratch :) If you do that, I will gladly help code review your pull requests :) You can find it on github: https://github.com/lvh/minitrue lvh
participants (3)
-
Laurens Van Houtven -
Phil Mayers -
Steve Chapel