Mailman 3 November 2004 - Twisted

[Twisted-Python] multiple connectTCP bindaddress problem
by Chris Laws 08 Nov '04

08 Nov '04

Looks like I'm moving to "OS specified addresses" anyway, as I can't get my current impementation working properly. Using the implementation where I increment the local port for each bainaddress arg to the connectTCP call, I was receiving "connectbinderror" on the client whenever I closed and restarted a Server. Interstingly I would also get this error if I started the client only (Ie. no servers running which it could connect to). In my logs I would see the initial connection attempt, a failure due to "connection refused". After a backoff time a second connection attempt would be made where again it would fail but with a "connectbinderror" - "address already in use". My client factory is inheriting from the ReconnectingClientFactory. Using netstat on Windows I could see the client side socket listed as "LISTENING". Is this right for a client side socket when not actually connected? Anyway I guess that when the ReconnectingClientFactory attempted to perform a retry it would see the address as "in-use" and fail with a bind error similar to my original problem where I was trying to bind to an address which was already listening. When I let the OS specify the address I don't see this problem. Surely it is performing the same functions though...just with OS specified IP:Port combination? My app is working now with OS specified ports, though I would still like to understand where I was going wrong and why? Chris

1 0

[Twisted-Python] multiple connectTCP bindaddress problem
by Chris Laws 08 Nov '04

08 Nov '04

I would like to specify my client ports for the previously mentioned reasons. We also use integration tools such as LAN message crackers to help us verify message contents. These use information such as "port" to determine the appropriate cracking method. If I let the OS determine the client's ports I will have to change the port information in our LAN message cracker everytime I restart the client application. Admittedly this will only be necessary during the integration phase, however I'm sure I'd get tired of manually changing port numbers in our cracker tool after a very short time.

2 1

[Twisted-Python] multiple connectTCP bindaddress problem
by Chris Laws 08 Nov '04

08 Nov '04

I probably have not explained myself very well. I'm only using a single protocol. The server side of my application is all fine, no problems with conflicts. The server side uses a single port. There are approx 30 machines each of which runs a separate instance of the server side application. They all bind the same port number on their local machine as a listen socket. My client side application will attempt to maintain a connection with all server side applications. The design allows for multiple client side applications to run (though not on the same machine). It's the client side I was concerned with. I agree that there will never be conflicts between client side applications. I'm working on a large project, by that I mean many application developed by disparate teams. There are many different applications performing different functions. My application is a very small part of the project and is used to monitor and display the process status of applications running on remote machines. It provides a custom way of starting/atopping allications in an ordered manner etc. Many of the project applications run on the same machine which my client application will be deployed on. It is these other applications which do not have well defined ports. Their ports are determined depending on who is logged in (port numbers are determined via some env setting). I wanted to make sure I used a specific port so that I could notify others that port "x" was now a well known port used by my application. To ensure I used a specific port I need to use the 'bindaddress' argument in the connectTCP call. I originally thought it would be nice to establish all the client application's outgoing connections as originating from the same IP:Port. Ie. client servers ------ 192.168.1.2:24000 / 192.169.1.1:31000 --------- 192.168.1.3:24001 \ ------ 192.168.1.4:24002 However from the post's I've read it seems that Windows does not support this functionality reliably. So, instead of using one port on the client side application to connect to all server ports, I will now need a unique port (on the client side) for each server that the client wishes to connect with. Hopefully design is clearer now and will help you help me. Is there a clearly better way to set up the client/sever connections?

2 1

[Twisted-Python] Re: Thank you!
by Stephen 08 Nov '04

08 Nov '04

1 0

[Twisted-Python] multiple connectTCP bindaddress problem
by Chris Laws 08 Nov '04

08 Nov '04

I only have about 30 servers which I connect the client(s) with. There are between 1 and 10 client applications which connect to the 30 servers, though it is very unlikely that more than two client side applications would be running at any one time. I have control over the ports I am using (within reason) so the client(s) can go on any port I choose. I simply wanted the clients to use the same local port when establishing their connections with the servers to reduce the likelyhood for conflicting use of ports. (We have a few conflicting uses of ports on this project so I wanted mine to use a few as possible). Anyway from your information I will go with the port-per-server implementation. 30 ports for the client isn't much anyway. Thanks for your help, Chris

2 1

[Twisted-Python] multiple connectTCP bindaddress problem
by Chris Laws 07 Nov '04

07 Nov '04

I'm in the process of building a twisted application which will be used to automate control of processes running on remote machines. The basic architecture is a client to many server configuration. 1. many (~30), twisted based, network server applications running on different machines, 2. a client side, twisted based, application which will be used to control and monitor the servers. This is a small component of a large project and as part of the project we have defined port ranges which applications can use so as not to interfere with other project applications. Therefore I need to remain within a certain port range. The client application uses a twisted client factory to establish the conection handlers. The machines I am running on have multiple interfaces so I am using the 'bindaddress' argument in the reactor.connecTCP method to force the client side connection estalishment to originate from a certain IP:port. I am calling connecTCP for each instance of a server which I need to conect with. I pass in a different server address each time but leave the 'bindaddress' the same. My understanding is that this should create a valid TCP connection to each different server IP:port address, all originating from the same client IP:Port. Maybe there is a fundamental flaw in my understanding of TCP but I think I can have multiple remote addresses connected to the same local IP:port address. Anyway this seems to be where my problem exists because when I define the same 'bindaddress' IP:Port tuple to each connectTCP call, I only seem to be able to connect to one remote server from the client application. To put this into more context I set up two servers, the first on 127.0.0.1:35000, the second on 192.168.1.2:36000. (The reason I'm not putting both connections on the same IP is that internal to the application I am tracking connections using the IP as the key in a dict, therefore I need unique IP strings). Then I setup the client bindaddress to use say 127.0.0.1:37000. (This problem does not seem to be related to the use of the 127... loopback address as I get the same result when I set the client to use the externally visable 192... address). If I don't specify the 'bindaddress' argument (Ie let the OS define the port etc) I can connect to multiple servers. However this means I don't conform to my port range usage requirements. I'm sure I'm missing something simple here. Any helpful ideas would be greatly appreciated. chris

4 5

[Twisted-Python] Deprecated function removal policy
by James Y Knight 07 Nov '04

07 Nov '04

So, I recently stumbled across t.i.base.ReactorBase.clientTCP/SSL/UNIX. They have been deprecated since Aug 2002. (version 0.99.0rc5 or so) Is there some policy for when deprecated functionality can be removed, or is it just going to stay forever? In addition, it would be really nice if every time something got deprecated, it got marked with the last version in which it was the recommended interface, and the date too, so less investigation needs to be done. James

2 1

[Twisted-Python] Re: Thank you!
by Stephen 07 Nov '04

07 Nov '04

1 0

[Twisted-Python] etag and last-modified
by Jacob Friis 06 Nov '04

06 Nov '04

I have a script that downloads multiple rss/atom feeds via Feedparser. The script uses twisted.internet but the developer tells me there is no way to use etag and last-modified with twisted. Instead I'll let Feedparser do the download and use twisted for threads. What is the maximum pool size I can use? Thanks, Jacob

4 7

[Twisted-Python] Server with several outgoing connections
by Roland Hedberg 06 Nov '04

06 Nov '04

Hi! I'm new to Twisted and I have tried to read and understand the intricate sides of it. And have found as so many other that the learning curve pretty steep, especially I guess if one comes from being a C programmer :-) Anyway, this is my problem: I'm building a system that consists of a number of nodes, each node acting as a receiver and transmitter of messages. The protocol used between the nodes might be a message passing system like xmpp or something else. The number of nodes are dynamic and might change at any time. Plus there is no way by which you can know when the system is started how many nodes there will be. Also nodes might receive messages from any number of nodes and be expected to transmit the messages to several nodes. Writing the 'server' part of a node is easy, plenty of examples of that exists on the net. Writing the 'client' side is not so easy and here I guess my C legacy is hampering me. So I'd like someone to give me, or point me, to an example on how to dynamically create new connections to receivers. I first though these connections should be shortlived that is 'open the connection, send the message, receive the ACK and close the connection' but now I think it might be wiser to keep the connections as long as possible. That is, until that receiver disappears. Another side of this is that the influx of messages might be higher than the possible output, that is each node has to keep queues for messages not yet sent and messages sent but not ack'ed. It might also happen that recivers of messages are not accessible in which case the message has to be queue until the receiver pops up again. The system is not allowed to drop messages. Note, that each incomming message might have several receivers, so one incomming message might end up in the sendqueue of several receivers. And here in lies one of the problems, I guess I have to keep a 'centrally' managed queue, that should then be notified on the change of status of a message on route to a recevier. How should I implement this ? --Roland

10 15