[Twisted-Python] multiple connectTCP bindaddress problem
I'm in the process of building a twisted application which will be used to automate control of processes running on remote machines. The basic architecture is a client to many server configuration. 1. many (~30), twisted based, network server applications running on different machines, 2. a client side, twisted based, application which will be used to control and monitor the servers. This is a small component of a large project and as part of the project we have defined port ranges which applications can use so as not to interfere with other project applications. Therefore I need to remain within a certain port range. The client application uses a twisted client factory to establish the conection handlers. The machines I am running on have multiple interfaces so I am using the 'bindaddress' argument in the reactor.connecTCP method to force the client side connection estalishment to originate from a certain IP:port. I am calling connecTCP for each instance of a server which I need to conect with. I pass in a different server address each time but leave the 'bindaddress' the same. My understanding is that this should create a valid TCP connection to each different server IP:port address, all originating from the same client IP:Port. Maybe there is a fundamental flaw in my understanding of TCP but I think I can have multiple remote addresses connected to the same local IP:port address. Anyway this seems to be where my problem exists because when I define the same 'bindaddress' IP:Port tuple to each connectTCP call, I only seem to be able to connect to one remote server from the client application. To put this into more context I set up two servers, the first on 127.0.0.1:35000, the second on 192.168.1.2:36000. (The reason I'm not putting both connections on the same IP is that internal to the application I am tracking connections using the IP as the key in a dict, therefore I need unique IP strings). Then I setup the client bindaddress to use say 127.0.0.1:37000. (This problem does not seem to be related to the use of the 127... loopback address as I get the same result when I set the client to use the externally visable 192... address). If I don't specify the 'bindaddress' argument (Ie let the OS define the port etc) I can connect to multiple servers. However this means I don't conform to my port range usage requirements. I'm sure I'm missing something simple here. Any helpful ideas would be greatly appreciated. chris
Chris Laws wrote:
My understanding is that this should create a valid TCP connection to each different server IP:port address, all originating from the same client IP:Port. Maybe there is a fundamental flaw in my understanding of TCP but I think I can have multiple remote addresses connected to the same local IP:port address.
This is a generic BSD socket API question that has nothing to do with Twisted or Python. Connections are identified by (srcIP, srcPort, dstIP, dstPort), yes. The protocol allows what you are asking. But of course even if the protocol allows something, not all implementations are not required to allow you to do it. In this case, the behaviour is selectable, with the default being do not allow it.
Anyway this seems to be where my problem exists because when I define the same 'bindaddress' IP:Port tuple to each connectTCP call, I only seem to be able to connect to one remote server from the client application.
Try "man 2 socket", SO_REUSEADDR Indicates that the rules used in validating addresses supplied in a bind(2) call should allow reuse of local addresses. For PF_INET sockets this means that a socket may bind, except when there is an active listening socket bound to the address. When the listening socket is bound to INADDR_ANY with a specific port then it is not possible to bind to this port for any local address.
Thanks for the information. I guess I see two ways forward. The first is to assign a different port in the bindaddress argument for each server I want to connect with. Not hard to refactor in... not really what I wanted but I can live with that. The second option would be to set the SO_REUSEADDR option "somewhere" and maintain my original implementation. I've googled around and found that there is a Python way of setting the SO_REUSEADDR option within asynccore 'set_reuse_addr'. I have not found anything similar for Twisted though. Is there some similar Python call within the Twisted framework which would allow me to set this option when setting up connection handlers with the reactor?
I've had a browse around the twisted source and it seems that SO_REUSEADDR is only set as a socket option when the platform is posix. I'm using a TCP connection which I think eventually uses twisted.internet.tcp.Port. In there it creates a socket and assigns the SO_REUSEADDR option if 'posix'. I can't find any toplevel twisted api call which would let me set the same option for a Windows platform machine. I thought this functionality was supported by Windows. However, explicitly enabling this option for posix makes me assume that this functionality is not supported by Windows (Ie non-support was found through some testing?). Is this the case? Is it not supported by Windows, or just poorly supported by Windows, hence not used by Twisted? Regards, Chris
On Mon, 2004-11-08 at 12:00 +1100, Chris Laws wrote:
I'm using a TCP connection which I think eventually uses twisted.internet.tcp.Port. In there it creates a socket and assigns the SO_REUSEADDR option if 'posix'.
connectTCP doesn't use tcp.Port, tcp.Port is internal code of listening (server) sockets.
Is this the case? Is it not supported by Windows, or just poorly supported by Windows, hence not used by Twisted?
It has some wonky behaviour, so I disabled it, but again, that was for server sockets. Your use case is "I want to have multiple TCP clients with same local IP, port", yes? This should only be necessary if: 1. You have tens of thousands of connections at once. 2. Your client has to be on a certain port to work, and you have multiple clients from same machine. If you are designing a new protocol doing this is a bad idea. Which of these is it in your case?
On Mon, Nov 08, 2004 at 12:00:52PM +1100, Chris Laws wrote: [...]
I'm using a TCP connection which I think eventually uses twisted.internet.tcp.Port. In there it creates a socket and assigns the SO_REUSEADDR option if 'posix'. [...] Is this the case? Is it not supported by Windows, or just poorly supported by Windows, hence not used by Twisted?
I think it's because it has confusingly different behaviour -- there's a brief note at http://www.advogato.org/article/672.html about this. -Andrew.
participants (4)
-
Andrew Bennetts -
Chris Laws -
Itamar Shtull-Trauring -
Tommi Virtanen