[Twisted-Python] twistd question
Hello, I have a small design problem I'm trying to get my head around regarding twistd with respect to lauching worker subprocesses. I have a requirement for every process on my server to be able to communicate to an existing message queuing system. The interface to this message queuing system requires that all processes be a "server" so that they can receive async messages. I have that all working fine and have the interface to the external system wrapped up in a nice service, as it us helpful to get the startService and stopService calls for properly initializing and shutting down the session to the message queuing system. As I have a mixture of a few long running requests and many short running requests that could be sent to the main server process, I would like to design my main service such that it can handle the short requests directly and spawn other subprocessing to handle the long-running requests. Since each of those subprocessing needs to be connected to the message queue, I'd like to use twistd to run them and benefit from the application level abstractions for setting up the service. But I'm not entirely sure how to use twistd property when I may have multiple instances of the same .tac file running. Basically, I need to spawn worker subprocessing that during the time they are doing work are services so that they can communicate with the messaging subsystem. Does this make sense? Any thoughts on how it can be accomplished? Ross
I forgot to mention one other item. The worker subprocesses are not involved in generating a network reponse to the request. Basically, the request sent to the main service process is, "Please do this at your earliest convenience", the request is validated and the server response with "okay," then launches the subprocess. Ross Jekel writes:
On 1/25/06, Ross Jekel <ross@sourcelabs.com> wrote:
I don't really see why you want the subprocesses to be run with twistd. If all they do is do some computations that aren't given back to the client, they might as well just be blocking code that you kick off with a spawnProcess. Maybe you could shed a bit more light on what these subprocesses will actually be doing, and what sort of data they need to pass back to the main server, if any. -- Twisted | Christopher Armstrong: International Man of Twistery Radix | -- http://radix.twistedmatrix.com | Release Manager, Twisted Project \\\V/// | -- http://twistedmatrix.com |o O| | w----v----w-+
I forgot to mention one other item. The worker subprocesses are not involved in generating a network reponse to the request. Basically, the request sent to the main service process is, "Please do this at your earliest convenience", the request is validated and the server response with "okay," then launches the subprocess. Ross Jekel writes:
On 1/25/06, Ross Jekel <ross@sourcelabs.com> wrote:
I don't really see why you want the subprocesses to be run with twistd. If all they do is do some computations that aren't given back to the client, they might as well just be blocking code that you kick off with a spawnProcess. Maybe you could shed a bit more light on what these subprocesses will actually be doing, and what sort of data they need to pass back to the main server, if any. -- Twisted | Christopher Armstrong: International Man of Twistery Radix | -- http://radix.twistedmatrix.com | Release Manager, Twisted Project \\\V/// | -- http://twistedmatrix.com |o O| | w----v----w-+
participants (2)
-
Christopher Armstrong -
Ross Jekel