[Twisted-Python] stop conditions and deferToThread()
Hi As discussed in a previous thread http://twistedmatrix.com/pipermail/twisted-python/2009-May/019717.html a task put in its own thread via deferToThread() won't stop even though the reactor has stopped. It has to carry out its own check that the reactor is running and stop if that is not the case. My question is: Is it possible to use anything else than the reactor.running as stop condition in this way? In my current twisted application I would like to keep the reactor running but still have some deferToThread jobs stop if e.g. an exception occurs in one of the jobs. In the code example below I try to use my own global field, running, as stop condition, but it doesn't work. If I let bar() call reactor.stop() after the time.sleep(3) and use reactor.running in foo() it works, however. import time from twisted.internet import reactor from twisted.internet.threads import deferToThread running = True def foo(): while running: print "working" time.sleep(1) def bar(): time.sleep(3) print "stopping" running = False d1 = deferToThread(foo) d2 = deferToThread(bar) reactor.run() Best regards, Thomas Jakobsen
Thomas Jakobsen wrote:
Hi
As discussed in a previous thread
http://twistedmatrix.com/pipermail/twisted-python/2009-May/019717.html
a task put in its own thread via deferToThread() won't stop even though the reactor has stopped. It has to carry out its own check that the reactor is running and stop if that is not the case.
My question is: Is it possible to use anything else than the reactor.running as stop condition in this way? In my current twisted application I would like to keep the reactor running but still have some deferToThread jobs stop if e.g. an exception occurs in one of the jobs.
A threading.Event object? Can you run the jobs as sub-processes? Then you could just kill them.
I'm not sure this is the problem you're facing, but... On Wed, Jun 10, 2009 at 12:04:16PM +0200, Thomas Jakobsen wrote:
import time from twisted.internet import reactor from twisted.internet.threads import deferToThread
running = True
def foo(): while running: print "working" time.sleep(1)
def bar(): global running time.sleep(3) print "stopping" running = False
d1 = deferToThread(foo) d2 = deferToThread(bar)
reactor.run()
...does that make things run better?
On Wed, 10 Jun 2009 12:04:16 +0200, Thomas Jakobsen <thomas.jakobsen@alexandra.dk> wrote:
Hi
As discussed in a previous thread
http://twistedmatrix.com/pipermail/twisted-python/2009-May/019717.html
a task put in its own thread via deferToThread() won't stop even though the reactor has stopped. It has to carry out its own check that the reactor is running and stop if that is not the case.
deferToThread is optimized for short-running tasks. Thread creation costs are amortized over the lifetime of the process. This is because the intended use of deferToThread is to run functions which compute some result and then return, /not/ for functions which run forever (or at least until the process is ready to exit). You should try to find a way to re-arrange your code so that there is no need to check if the reactor is still running - just do one piece of work in a thread at a time. When your non-threaded application code gets the result back, if no one has asked it to shut down, then it can ask for another job to be executed.
My question is: Is it possible to use anything else than the reactor.running as stop condition in this way?
Also, I want to point out that while `reactor.running´ isn't private (since it doesn't start with an underscore, as Twisted's privacy policy dictates it would need to), it still isn't something you really want to be using. It's not part of any interface, which means there is no guarantee it will be provided by all reactors. It's also not really for you. It's a flag that the reactor uses to track its own internal state. Its meaning may not actually correspond to the meaning you'd like to assign to it. Whenever you're trying to do something like this (which I still don't recommend), you should create your own state tracking and use that instead. Jean-Paul
Thanks for the replies. In my particular case, adding a global running as Tim Allen pointed out, actually did it. Regards, Thomas
participants (4)
-
Jean-Paul Calderone -
Phil Mayers -
Thomas Jakobsen -
Tim Allen