[Twisted-Python] Problem exiting script that calls a func returning a called deferred
From searching and reading the list archives, I know the following comes up under various guises, and although I think I understand what's going on, I don't see a nice solution.
I'm trying to write a standalone script. It calls externalFunc, a function someone else wrote that returns a deferred. My script could look something like this: from twisted.internet import reactor from twisted.python import log if __name__ == '__main__': def ok(result): print "ok called with", result def nok(failure): print "nok called with", failure return failure def stop(x): reactor.stop() d = externalFunc() d.addCallback(ok) d.addErrback(nok) d.addBoth(stop) d.addErrback(log.err) reactor.run() And that works fine. Now suppose I want to do some testing in which I swap out the external function for a library I wrote myself (or for another implementation). In the testing library, externalFunc is simplified and uses defer.success() to return its result. In this case the script raises RuntimeError, "can't stop reactor that isn't running" because the deferred that comes back from externalFunc has already been called. So when I add the call/errbacks, they are fired right away and the reactor isn't running when the stop function tries to stop it. And script never exits because it then starts the reactor. At this point there are various contortions I can try. Catching the RuntimeError works, but the script never exits. Catching the RuntimeError and then calling sys.exit doesn't work because sys.exit raises SystemExit and that is caught in twisted/internet/defer.py _runCallbacks. Supposing I had more control, the calling function could create the deferred, pause it, pass it to externalFunc, add callbacks, then unpause. But that doesn't work either as the callbacks fire as soon as unpause is called, and you get the reactor not started error, followed by no exit as you then start the reactor. Testing reactor.running before trying to stop it in stop gets rid of the error, but the reactor is then started and not stopped. The only bulletproof way to deal with this that I've come up with is to call self.fireSystemEvent("shutdown") but that feels wrong. Several solutions seem possible with changes to Twisted: don't catch SystemExit; give reactor.stop a (default False) keyword arg to tell it to ignore a stopped reactor; stop deferreds from firing call/err backs if they are paused (IMO it's a bit non-paradigmatic to have call/errback events firing when the reactor isn't even running), etc. I suppose I'm just not doing things right. Can someone tell me how I should be doing this? Terry
Hello, Terry Jones wrote:
From searching and reading the list archives, I know the following comes up under various guises, and although I think I understand what's going on, I don't see a nice solution.
I'm trying to write a standalone script. It calls externalFunc, a function someone else wrote that returns a deferred. My script could look something like this:
from twisted.internet import reactor from twisted.python import log
if __name__ == '__main__': def ok(result): print "ok called with", result def nok(failure): print "nok called with", failure return failure def stop(x): reactor.stop()
d = externalFunc() d.addCallback(ok) d.addErrback(nok) d.addBoth(stop) d.addErrback(log.err) reactor.run()
And that works fine.
Now suppose I want to do some testing in which I swap out the external function for a library I wrote myself (or for another implementation). In the testing library, externalFunc is simplified and uses defer.success() to return its result.
In this case the script raises RuntimeError, "can't stop reactor that isn't running" because the deferred that comes back from externalFunc has already been called. So when I add the call/errbacks, they are fired right away and the reactor isn't running when the stop function tries to stop it.
And script never exits because it then starts the reactor.
did you try something like: def stop(x): reactor.callLater(0, reactor.stop) This ensures that the reactor is started before you tell it to stop. -- Amaury Forgeot d'Arc
participants (2)
-
Amaury Forgeot d'Arc -
Terry Jones