[Twisted-Python] Punching a firewall with Twisted?
Dear 'Twisted' experts! I have a quick question that I am sure one of you guys can answer easily: What I want to do is open an incoming port on the router that connects my computer to the Internet. More precisely, I want my Python app do that automatically. In other words, this is about punching a hole into a firewall. I know that this is possible because there are applications that do that. Deluge does this, and Deluge uses Twisted. So I have tried to find out how/where Deluge tells Twisted to do this, but the Deluge sources are quite sizey, and I haven't found that particular needle in the haystack yet. So, can one of you people enlighten me here? :) Thanks & have a nice day! Stefan
On Jun 17, 2010, at 8:37 AM, Stefan Reich wrote:
Dear 'Twisted' experts!
I have a quick question that I am sure one of you guys can answer easily:
What I want to do is open an incoming port on the router that connects my computer to the Internet. More precisely, I want my Python app do that automatically.
In other words, this is about punching a hole into a firewall.
I know that this is possible because there are applications that do that. Deluge does this, and Deluge uses Twisted. So I have tried to find out how/where Deluge tells Twisted to do this, but the Deluge sources are quite sizey, and I haven't found that particular needle in the haystack yet.
So, can one of you people enlighten me here? :)
Thanks & have a nice day! Stefan
I'm not sure about Deluge, but <http://divmod.org/trac/wiki/DivmodVertex> does some NAT hole-punching. It does it via sending UDP packets though, not by communicating directly with your router. Good luck; this is always an interesting problem ;). -glyph
Hi Glyph! (Wait, is that really a first name...? :))
I'm not sure about Deluge, but <http://divmod.org/trac/wiki/DivmodVertex> does some NAT hole-punching. It does it via sending UDP packets though, not by communicating directly with your router.
Yeah, I found that too... it looks quite interesting, but as you say, it is UDP-based. I know that there is a working TCP-solution. As I said, Deluge does it, and for example, the original Bittorrent client does it too. I actually verified this on my machine: With a trick, I can exploit these clients for my purposes. I open a server socket in my own program on port 15667. Then I start Bittorrent. It complains because it can't open the server socket - but it does open the firewall, and the firewall stays open as long as the application runs. So voila, I can now access my server from outside. It is actually open right now, you can try if you like: telnet 188.193.214.124 15667 :]
Good luck; this is always an interesting problem ;).
Yeah... it's actually something that begins to really annoy me. Why are we put behind these firewalls? And why is it so complicated to open ports on them? It shouldn't be. Let's change that. We have the means! Every internet user should be able to run a server. This is my credo! Cheers, Stefan
On Thursday 17 June 2010, Stefan Reich wrote:
Yeah, I found that too... it looks quite interesting, but as you say, it is UDP-based. I know that there is a working TCP-solution. As I said, Deluge does it, and for example, the original Bittorrent client does it too.
Maybe it uses UPNP to tell the router which ports to open? As far as I know, there are no hole-punching techniques for TCP like there are for UDP. So the only way to get a port forward is to configure the router, manually or through an API like UPNP. Bye, Maarten
On Jun 17, 2010, at 12:59 PM, Maarten ter Huurne wrote:
Maybe it uses UPNP to tell the router which ports to open?
As far as I know, there are no hole-punching techniques for TCP like there are for UDP. So the only way to get a port forward is to configure the router, manually or through an API like UPNP.
Yes, many bittorrent clients use UPNP and/or NAT-PMP to open a hole in the firewall. This will work for most users of home routers, but not other NAT scenarios. I'm just waiting for the kernel patch which makes socket()/bind() automatically send a UPNP/NAT-PMP to the router to open the port. :) James
On Jun 17, 2010, at 9:37 AM, Stefan Reich wrote:
I'm not sure about Deluge, but <http://divmod.org/trac/wiki/DivmodVertex> does some NAT hole-punching. It does it via sending UDP packets though, not by communicating directly with your router.
Yeah, I found that too... it looks quite interesting, but as you say, it is UDP-based. I know that there is a working TCP-solution. As I said, Deluge does it, and for example, the original Bittorrent client does it too.
For what it's worth, Vertex includes an *implementation* of TCP, on top of UDP instead of IP. You can run any Twisted stream-based protocol with Vertex, you don't need to send UDP datagrams around. (In fact, Vertex _only_ provides a stream-based transport; the UDP part is an implementation detail. If possible, vertex will try to use TCP first, and could be modified to use a techniques such as the ones described in <http://www.brynosaurus.com/pub/net/p2pnat/> section 4 to speak real TCP instead.)
2010/6/17 Stefan Reich <wertiges.produkt@googlemail.com>:
Dear 'Twisted' experts!
I have a quick question that I am sure one of you guys can answer easily:
What I want to do is open an incoming port on the router that connects my computer to the Internet. More precisely, I want my Python app do that automatically.
In other words, this is about punching a hole into a firewall.
I know that this is possible because there are applications that do that. Deluge does this, and Deluge uses Twisted. So I have tried to find out how/where Deluge tells Twisted to do this, but the Deluge sources are quite sizey, and I haven't found that particular needle in the haystack yet.
So, can one of you people enlighten me here? :)
I don't know if this is Deluge's method, but sounds interesting: http://samy.pl/natpin/?port=80 If you don't want see deluge code, maybe you want use a sniffer. Wireshark is great for these things ;-) http://www.wireshark.org/ Regards, Javi
lasizoillo wrote:
I don't know if this is Deluge's method, but sounds interesting: http://samy.pl/natpin/?port=80
Hm. That's another kind of hack, and it didn't work for me.
If you don't want see deluge code, maybe you want use a sniffer. Wireshark is great for these things ;-) http://www.wireshark.org/
Wireshark! What a nice tool that wasn't on my radar before. Thanks for this, it really helped. Using Wireshark, I found out that both Bittorrent and Deluge talk to the gateway via UPNP for opening the port. It all works over HTTP, so it's really easy to see what's going on. These are the steps: 1. Find out the gateway IP (192.168.0.1 or some such) 2. GET http://{gateway}/desc.xml. This makes the gateway print some information about itself (make & model and so on). Not sure why this is needed. Maybe the protocol is different for different gateways. Or to make sure that there is a gateway at all. Or maybe there is some detail information in there that is needed for the next steps. 3. Also sent to the gateway: POST /upnp/control3 with some XML/SOAP data containing a command called AddPortMapping. It looks really easy, like you could reprogram this in a few lines, with a SOAP library, or even just manual XML. There's also a command called DeletePortMapping for proper cleanup. And it seems like Twisted may actually not even be involved in Deluge's port opening code. Ah. I'm really quite satisfied. I heard that there are tricks to open firewalls (Skype's been doing this for ages), but I didn't know that there is such an elegant solution. Cheers and thanks for the hints, Stefan
Coincidentally there's been some talk about this over on tahoe-dev@tahoe-lafs.org: http://tahoe-lafs.org/pipermail/tahoe-dev/2010-June/004469.html Tahoe-LAFS is a distributed storage system which uses Twisted. Some people think there aren't enough routers that implement UPnP well enough to make it worthwhile. Shawn Willden posted some code to the tahoe-dev mailing list showing how to do it using "miniupnp". jrydberg posted a link to the Tahoe-LAFS trac ticket to his Python NAT-PMP implementation. Regards, Zooko
On Jun 17, 2010, at 3:16 PM, Zooko O'Whielacronx wrote:
Some people think there aren't enough routers that implement UPnP well enough to make it worthwhile
Doesn't *every* home router purchased in the last 5 years support one of UPnP or NAT-PMP? That's been my experience, at least. James
On 2010-06-17, James Y Knight wrote:
On Jun 17, 2010, at 3:16 PM, Zooko O'Whielacronx wrote:
Some people think there aren't enough routers that implement UPnP well enough to make it worthwhile
Doesn't *every* home router purchased in the last 5 years support one of UPnP or NAT-PMP? That's been my experience, at least.
Not all of them. More to the point, I'm not aware of any corporate routers that support these things. It's too much of a security risk to be able to have arbitary machines on your LAN set up port forwards. -- Regards, Stephen Thorne Development Engineer Netbox Blue
participants (7)
-
Glyph Lefkowitz -
James Y Knight -
lasizoillo -
Maarten ter Huurne -
Stefan Reich -
Stephen Thorne -
Zooko O'Whielacronx