[Twisted-Python] using a reactor in a loop
Hello, I am trying to use a reactor in a loop. The application will go through the loop twice then hang when trying to stop the protocol. Here's my loop: def main(): factory = Factory() factory.numProtocols = 0 factory.protocol = AqapProtocol while True: print 'running server' # do other work ... reactor.listenTCP(48555, factory) reactor.run() # stopped by protocol Here's how the protocol stops the reactor: def connectionLost(self, reason): self.factory.numProtocols -= 1 if not self.factory.numProtocols: print 'stopping reactor' reactor.stop() print 'reactor stopped' I see the 'reactor stopped' get printed out twice through the loop but I don't see 'running server' get printed out a third time. So, reactor.run() is not returning. I must be missing something or going at this completely wrong. Any suggestions? Thanks, Tom
On Thu, 1 Mar 2007 09:21:50 -0800, Tom Brown <brown@esteem.com> wrote:
Hello,
I am trying to use a reactor in a loop. The application will go through the loop twice then hang when trying to stop the protocol. Here's my loop:
[snip]
I see the 'reactor stopped' get printed out twice through the loop but I don't see 'running server' get printed out a third time. So, reactor.run() is not returning. I must be missing something or going at this completely wrong. Any suggestions?
The simple thing you're missing is that reactors can't be restarted. :) One way you can restructure your program to avoid needing to do this is to put the body of your loop (minus reactor.run()) into a function which returns a Deferred and to replace the call to reactor.stop() with code which fires that Deferred. You can set up callbacks on this Deferred to execute the whole thing again if necessary. Jean-Paul
On Thursday 01 March 2007 09:32, Jean-Paul Calderone wrote:
On Thu, 1 Mar 2007 09:21:50 -0800, Tom Brown <brown@esteem.com> wrote:
Hello,
I am trying to use a reactor in a loop. The application will go through the loop twice then hang when trying to stop the protocol. Here's my loop:
[snip]
I see the 'reactor stopped' get printed out twice through the loop but I don't see 'running server' get printed out a third time. So, reactor.run() is not returning. I must be missing something or going at this completely wrong. Any suggestions?
The simple thing you're missing is that reactors can't be restarted. :) One way you can restructure your program to avoid needing to do this is to put the body of your loop (minus reactor.run()) into a function which returns a Deferred and to replace the call to reactor.stop() with code which fires that Deferred. You can set up callbacks on this Deferred to execute the whole thing again if necessary.
I read through the deferreds documentation and I'm thoroughly confused on how to set this up. I tried to simplify the problem with a simple function: In [60]: def foo(): ....: print 'foo' ....: return Deferred() ....: In [61]: d = foo() foo In [62]: d.addCallback(foo) Out[62]: <Deferred at 0xb786792cL> So, now I'm lost as to what to do from here. Any suggestions would be appreciated. Thanks, Tom
On Thu, 1 Mar 2007 13:28:04 -0800, Tom Brown <brown@esteem.com> wrote:
On Thursday 01 March 2007 09:32, Jean-Paul Calderone wrote:
On Thu, 1 Mar 2007 09:21:50 -0800, Tom Brown <brown@esteem.com> wrote:
Hello,
I am trying to use a reactor in a loop. The application will go through the loop twice then hang when trying to stop the protocol. Here's my loop:
[snip]
I see the 'reactor stopped' get printed out twice through the loop but I don't see 'running server' get printed out a third time. So, reactor.run() is not returning. I must be missing something or going at this completely wrong. Any suggestions?
The simple thing you're missing is that reactors can't be restarted. :) One way you can restructure your program to avoid needing to do this is to put the body of your loop (minus reactor.run()) into a function which returns a Deferred and to replace the call to reactor.stop() with code which fires that Deferred. You can set up callbacks on this Deferred to execute the whole thing again if necessary.
I read through the deferreds documentation and I'm thoroughly confused on how to set this up. I tried to simplify the problem with a simple function:
In [60]: def foo(): ....: print 'foo' ....: return Deferred() ....:
In [61]: d = foo() foo
In [62]: d.addCallback(foo) Out[62]: <Deferred at 0xb786792cL>
So, now I'm lost as to what to do from here. Any suggestions would be appreciated.
Something needs to call that Deferred back for anything further to happen (though what will happen next is a TypeError, since the callback, `foo', doesn't accept any arguments, and all Deferred callbacks need to accept at least one argument. ;) For example:
def foo(result): ... print 'Got a result:', result ... d = Deferred() d.addCallback(foo) <Deferred at 0xB7D26F4CL> d.callback("Hello, world") Got a result: Hello, world
Or to set up a (terrible, infinite, actually-infinitely-recursive) loop:
def bar(lastResult=None): ... print 'Previous result was:', lastResult ... d = Deferred() ... d.addCallback(bar) ... d.callback(str(lastResult) + "bar") ...
(Slightly) More realistically, you might have something like this:
class NoOp(Protocol): ... def __init__(self, onConnect): ... self.onConnect = onConnect ... def connectionMade(self): ... self.transport.loseConnection() ... self.onConnect.callback(None) ... def baz(ignored=None): ... d = Deferred() ... p = NoOp(d) ... f = ClientFactory() ... f.protocol = lambda: p ... reactor.connectTCP(host, port, f) ... d.addCallback(baz)
Which is similar to the previous example, in that it sets up a loop with a Deferred for an asynchronous operation (connecting to a remote host), but won't lead to infinite recursion, and might actually resemble what you want to do. :) Jean-Paul
Hi Tom, On Thu, 01 Mar 2007 11:21:50 -0600, Tom Brown <brown@esteem.com> wrote:
Hello,
I am trying to use a reactor in a loop. The application will go through the loop twice then hang when trying to stop the protocol. Here's my loop:
The reactor provides an event loop already, so I am confused as to why you are trying to run it inside another loop. Based on the code you have posted here, I can't ascertain what your goal is. Generally speaking, you'd be better served by using the reactor's loop instead of your own, and stopping/starting your listeners, rather than the reactor itself. Also, stopping the reactor from within a protocol is probably not ideal. That's sort of like telling your OS to shut down when you're done saving a file ;) Hope this helps, L. Daniel Burr
On Thursday 01 March 2007 09:42, L. Daniel Burr wrote:
Hi Tom,
On Thu, 01 Mar 2007 11:21:50 -0600, Tom Brown <brown@esteem.com> wrote:
Hello,
I am trying to use a reactor in a loop. The application will go through the loop twice then hang when trying to stop the protocol. Here's my loop:
The reactor provides an event loop already, so I am confused as to why you are trying to run it inside another loop. Based on the code you have posted here, I can't ascertain what your goal is.
Generally speaking, you'd be better served by using the reactor's loop instead of your own, and stopping/starting your listeners, rather than the reactor itself.
Also, stopping the reactor from within a protocol is probably not ideal. That's sort of like telling your OS to shut down when you're done saving a file ;)
Thanks for your reply. My goal is this: 1) do specific tasks 2) run the server and wait for a connection 3) when a connection is made, do tasks as instructed by the client in the AqapProtocol code. 4) when connection is lost, close server and go back to 1) Shutting down the server is not necessary if I could get past the reactor.run(). A previous post suggested I use deferreds. So, that is probably what I need to. I just need to learn how to use them. Thanks, Tom
Tom Brown wrote:
My goal is this: 1) do specific tasks 2) run the server and wait for a connection 3) when a connection is made, do tasks as instructed by the client in the AqapProtocol code. 4) when connection is lost, close server and go back to 1)
Shutting down the server is not necessary if I could get past the reactor.run(). A previous post suggested I use deferreds. So, that is probably what I need to. I just need to learn how to use them.
There's really good documentation about twisted and its event-driven programming model, there you'll find what you're looking for. in twisted you never "get past" the reactor.run() (until you quit the application), everything is done inside its loop. so when using twisted you should rather consider this: 1) run the event-loop 2) do specific tasks (if you need to complete long running tasks, you could use deferToThread or a pool of threads, to keep your event-loop responsive ) 3) setup listener for connections (now you have your server) 4) when a connection is made, do tasks as instructed by the client in the AqapProtocol code. 5) if really necessary, stop listening for/refuse connections without stopping the eventloop, but maybe it's better to return a "service suspended, come back later" kind of message 6) goto 2) AFAIK twisted doesn't support restarting the reactor, there're some issues with cleaning up the state. But when you get used to it, you'll never feel the need to do that anyway, which might be the main reason why it's neither supported nor really discussed. You can do anything you want while the reactor is running. Johann
Thanks, Tom
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
participants (4)
-
Jean-Paul Calderone -
Johann Borck -
L. Daniel Burr -
Tom Brown