RE: [Twisted-Python] What is the "Twisted" way to kill stuck child process
Again, I could be taking the wrong approach: Here is the scenerao: 1) Why the pooling Each compute node can run 2 processes at the same time (big processes) The compute node must accept more jobs then it can run, and hold the waiting tasks in a queue. (logically similar problem to database connections, except, compute connections are used instead). 2) why not spawnProcess The code said that spawnprocess is broken on windows. (I could be looking in the wrong place, twisted is a big codebase :-) I fell back to using popen2 in a thead managed by the "thread pool" I am always open to refactoring a design. Windows is my prime platform, but I also must support the UNIX platforms. Hope this helps clairify what I am doing (not saying that I am taking the correct approach with twisted, but I am trying and willing to change to do it the twisted way :-) Thank you very much, Mike ---------------------------------------------------------------- Michael Schneider Senior Software Engineering Consultant UGS PLM Solutions - an EDS Company "The Greatest Performance Improvement Is the transitioning from a non-working state to the working state"
-----Original Message----- From: twisted-python-admin@twistedmatrix.com [mailto:twisted-python-admin@twistedmatrix.com]On Behalf Of Jp Calderone Sent: Thursday, February 12, 2004 10:27 AM To: twisted-python@twistedmatrix.com Subject: Re: [Twisted-Python] What is the "Twisted" way to kill stuck child process
On Thu, Feb 12, 2004 at 08:57:07AM -0500, Schneider, Michael wrote:
Hello All,
I am making real progress with twisted. Thank you very much for the great project.
I am creating a process launcher that runs on SGI, SUN, IBM AIX, HPUX, and windows 2000 and XP.
I modified the adbapi.py file to pool jobs. It works great.
Problem: Sometimes the launched c program will hang. (standard exe file). I want to launch a process with timeout value.
Ex. I know that task X usually take 1200 seconds to run, but it will sometimes hang. If the exe has not finished running after 1500 seconds, I want to kill the job and notify the caller that their job timed out.
What are some options for implementing this with twisted.
NOTE: each process is launched in its own thread, so killing the thread is an option is that is required.
I don't understand this. Are you using reactor.spawnProcess() to start these processes? If not, you should be. If so, it should be called from the main thread, and the process can be killed with the usual means, os.kill(pid). The PID is available, though I forget exactly where, probably as processProtocol.transport.pid. You can use reactor.callLater to set up a simple timeout so that processes are killed if they run for too long.
Jp
On Thu, Feb 12, 2004 at 11:03:52AM -0500, Schneider, Michael wrote:
Again, I could be taking the wrong approach:
Here is the scenerao:
1) Why the pooling Each compute node can run 2 processes at the same time (big processes) The compute node must accept more jobs then it can run, and hold the waiting tasks in a queue. (logically similar problem to database connections, except, compute connections are used instead).
2) why not spawnProcess The code said that spawnprocess is broken on windows. (I could be looking in the wrong place, twisted is a big codebase :-)
I fell back to using popen2 in a thead managed by the "thread pool"
I am always open to refactoring a design. Windows is my prime platform, but I also must support the UNIX platforms.
Hope this helps clairify what I am doing (not saying that I am taking the correct approach with twisted, but I am trying and willing to change to do it the twisted way :-)
I'm not certain of the status of spawnProcess on Windows. If it really doesn't work, I suppose popen2 is about all you can do. Since you're mainly developing on Windows, I doubt I can offer much more assistance, having never written any multiprocess Windows apps. If popen2 lets you get PIDs on Windows, and os.kill() works with those PIDs on Windows, I think you should still be able to timeout processes fairly easily. If they don't, maybe someone with more Windows experience can chime in. Jp
os.kill() isn't available on Windows, at least not in my version of Python 2.3. I wrote a C extension that would allow me to shut down some programs, but I had to get a handle to a process' window before I could get its PID. Since I knew the class names and/or window caption text of the windows in the applications I wanted to close, I stopped when I figured out how to find the window handles and didn't look into it any more. If you can get the PID from some other source, you can write an extension to use OpenProcess and TerminateProcess to get a handle to the process and kill it. Let me know if you'd like some code samples and I can go dig them up. Jp Calderone wrote:
If popen2 lets you get PIDs on Windows, and os.kill() works with those PIDs on Windows, I think you should still be able to timeout processes fairly easily. If they don't, maybe someone with more Windows experience can chime in.
On Thu, 2004-02-12 at 11:50, Jp Calderone wrote:
I'm not certain of the status of spawnProcess on Windows. If it really doesn't work, I suppose popen2 is about all you can do.
Since you're mainly developing on Windows, I doubt I can offer much more assistance, having never written any multiprocess Windows apps.
If popen2 lets you get PIDs on Windows, and os.kill() works with those PIDs on Windows, I think you should still be able to timeout processes fairly easily. If they don't, maybe someone with more Windows experience can chime in.
If you want to use spawnProcess on windows, you must use the win32 reactor. This has some limiting effects (but then, so does windows itself). Don't use popen2 with Twisted - the SIGCHLD handler that twistd installs will cause conflicts and crashes.
participants (4)
-
Alan McIntyre -
Glyph Lefkowitz -
Jp Calderone -
Schneider, Michael