[Twisted-Python] UDP vs TCP for bittorrent-style file transfer
Hi list, I'd like to revive a topic touched on in the past. I'm playing with python socket programming and decided I want to make a simplistic bittorrent clone that uses UDP instead of TCP. My main reasons for the protocol choice are (1) I don't need guaranteed in-order delivery, and (2) I need to get around NAT restrictions without making the user manually forward ports. Couple questions... 1. Are those good reasons to ditch TCP, or am I being an idiot? (Always a possibility.) 2. Even though I don't need TCP's in-order guarantee, I still need an ACKing mechanism to make sure all the packets eventually arrive. Is there any (simple) code written that accomplishes this with UDP?
On Sun, 21 Sep 2008 20:35:42 -0400, "Z. S. O." <tiredashell@gmail.com> wrote:
Hi list, I'd like to revive a topic touched on in the past. I'm playing with python socket programming and decided I want to make a simplistic bittorrent clone that uses UDP instead of TCP. My main reasons for the protocol choice are (1) I don't need guaranteed in-order delivery, and (2) I need to get around NAT restrictions without making the user manually forward ports. Couple questions...
1. Are those good reasons to ditch TCP, or am I being an idiot? (Always a possibility.)
It's not easier to get UDP through NATs. It's just as hard or harder.
2. Even though I don't need TCP's in-order guarantee, I still need an ACKing mechanism to make sure all the packets eventually arrive. Is there any (simple) code written that accomplishes this with UDP?
Jean-Paul
It's not easier to get UDP through NATs. It's just as hard or harder.
It must be a widely held error, then, because it's what I hear every time the subject arises. My understanding was that since UDP doesn't have the concept of "streams," most NATs will allow all UDP packets through to a given port one it is first hole-punched. Nat traversal aside, I'm also concerned about performance, and since TCP doesn't make in-order delivery optional, I'm still tempted to stick with UDP...
On Sun, 21 Sep 2008 21:06:12 -0400, "Z. S. O." <tiredashell@gmail.com> wrote:
It's not easier to get UDP through NATs. It's just as hard or harder.
It must be a widely held error, then, because it's what I hear every time the subject arises. My understanding was that since UDP doesn't have the concept of "streams," most NATs will allow all UDP packets through to a given port one it is first hole-punched.
Nat traversal aside, I'm also concerned about performance, and since TCP doesn't make in-order delivery optional, I'm still tempted to stick with UDP...
For a bittorrent-like application, you're also not particularly concerned with *latency*, just throughput. Allowing out-of-order delivery primarily reduces latency, it doesn't do a whole lot for throughput. Jean-Paul
For a bittorrent-like application, you're also not particularly concerned with *latency*, just throughput. Allowing out-of-order delivery primarily reduces latency, it doesn't do a whole lot for throughput.
That's a good point, I didn't consider that. Latency is only really important for streaming media and gaming, so this probably is a bad idea after all. Well anyway thanks for the input, I appreciate it.
On Sep 21, 2008, at 9:06 PM, Z. S. O. wrote:
It's not easier to get UDP through NATs. It's just as hard or harder.
It must be a widely held error, then, because it's what I hear every time the subject arises. My understanding was that since UDP doesn't have the concept of "streams," most NATs will allow all UDP packets through to a given port one it is first hole-punched.
Yes, that's usually true, especially for home NATs. Some NATs do actually track UDP conversations between endpoints just like TCP streams, though. Traversing a NAT with UDP is /usually/ easier, when you have multiple endpoints you wish to hear from. But don't forget about the other kinds of NATs too, which you'll also have to implement support for. That's what makes it hard: implementing things multiple times for all the different network topologies.
Nat traversal aside, I'm also concerned about performance, and since TCP doesn't make in-order delivery optional, I'm still tempted to stick with UDP...
Not entirely true. In order delivery is preserved to your application, but out-of-order delivery between the host endpoints is supported. Read up on the TCP "selective ACK" feature [1], which basically all OSes made since 1998 or so support. So a missed packet or two doesn't actually require all later data to always be retransmitted. James [1] http://www.ietf.org/rfc/rfc2018.txt
On 01:06 am, tiredashell@gmail.com wrote:
It's not easier to get UDP through NATs. It's just as hard or harder.
It must be a widely held error, then, because it's what I hear every time the subject arises. My understanding was that since UDP doesn't have the concept of "streams," most NATs will allow all UDP packets through to a given port one it is first hole-punched.
If you are a piece of software behind a NAT, and you want to accept a connection from a peer, there are three things you can do: 1. Require some explicit configuration from the user; ask them to punch a hole through their NAT and tell you what port to use. This is by far the easiest for you, but considered unacceptable since its default configuration will usually be broken. 2. Use a protocol like UPnP or Nat-PMP to get around the user's router. This will work on a minority of deployed endpoints, but when it does work it will work great, and your application can just make regular TCP connections. 3. Punch a hole, abusing the router's NAT routing rules in order to get it to think it's making a connection when it is actually accepting one. There are about six different ways to do this, but you can do it with UDP *or* TCP. Different routers support different schemes, and I don't just mean "some support TCP, some support UDP": UDP port mapping _usually_ works a certain way, but there are exceptions. You should at least read the papers referenced here: http://midcom- p2p.sourceforge.net/ In order to build a truly robust application you will need to do all of the above. Even if you do #2 and #3 perfectly, there are a surprising percentage of home routers (say, 5-10%) which are just buggy and won't route your traffic properly to another NAT no matter what you do, so you always need to provide a "just tell the NAT what to do manually" option. (Or, forward your traffic through some known-to-work intermediary.)
Nat traversal aside, I'm also concerned about performance, and since TCP doesn't make in-order delivery optional, I'm still tempted to stick with UDP...
TCP will, in general, "perform" better than UDP, because TCP can cheat whereas UDP can't. Your ISP can (and probably does, according to rumors I've heard from people who have worked on such things) deploy "smart" routers which can prioritize traffic using the additional information present in TCP headers which UDP doesn't provide. Plus, there are things that your operating system is doing (as James already mentioned) that are extensions to the core TCP spec. If you don't understand _at least_ enough about TCP to have implemented it yourself once, then you probably don't understand enough to make the decision to use UDP for performance reasons. I am saying this because it was at the point where I actually implemented TCP myself that I gained this insight :). I started a project a while ago to try to abstract away all this NAT- traversal nonsense and present an API for making a connection to a peer- to-peer user over whatever methods are available. It's definitely in need of some maintenance, but if you're already using Twisted I think you'll find it easier to get started with this than to build your own P2P framework from the ground up: http://divmod.org/trac/wiki/DivmodVertex Good luck!
participants (4)
-
glyph＠divmod.com -
James Y Knight -
Jean-Paul Calderone -
Z. S. O.