Re: [Twisted-Python] Synchronization techniques
Hello Brian: I have just recently started reading this thread.
1. The interactive python interpreter is a completely >synchronous universe - getting the reactor running in >this context seems like a hack. The only way I have >seen this done is by running the reactor in a different thread. The problem with this is that it is inevitable that you end up wanting to do things with Deferreds in the main thread where user code is running. But, as I understand it, Twisted is not thread safe, so at that point, you are playing with (threaded) fire.
I recented subscribed to this mailing list to ask questions about integrating Stackless with Twisted.... I execute Stackless and Twisted in their own threads. Otherwise when the Twisted reactor blocks, Stackless tasklets (read threads) that could otherwise run, would be blocked too. I start the Stackless thread with a reactor.callInThread(). Because of thread safety, I use the callFromThread() method to make Twisted calls. Also for safety and performance, I make the threads communicate through a deque (I took Phil Mayer's suggestion). As for deferreds. Essentially I make the deferreds and Stackless threads communicate through channels. So Stackless tasklets block until there is input. There are still a few issues and I am still conducting tests, However things seem to be working... One of the Stackless members, Carlos de Paula has started a repository that includes Twisted/Stackless examples. There are examples using the Perspective Broker and callInThread/callFromThread. http://code.google.com/p/stacklessexamples/wiki/StacklessTwisted Cheers, Andrew ____________________________________________________________________________________ No need to miss a message. Get email on-the-go with Yahoo! Mail for Mobile. Get started. http://mobile.yahoo.com/mail
On 11:25 am, andrewfr_ice@yahoo.com wrote:
I execute Stackless and Twisted in their own threads. Otherwise when the Twisted reactor blocks, Stackless tasklets (read threads) that could otherwise run, would be blocked too.
Stackless tasklets that "want to run" ought to be running due to events transpiring. One kind of event is "time passes" (reactor.callLater), other types of events are user interaction (like callbacks in a UI such as GTK) or I/O events (dataReceived, connectionLost, et. al.) There should be no need for native threads in a Twisted+Stackless universe. Even if all your tasklets constantly want to be doing work, they should be able to do that via a tasklet scheduler that is always running with a 0 callLater timeout. Do you think this would be possible?
On 4/6/07, glyph@divmod.com <glyph@divmod.com> wrote:
On 11:25 am, andrewfr_ice@yahoo.com wrote:
I execute Stackless and Twisted in their own threads. Otherwise when the Twisted reactor blocks, Stackless tasklets (read threads) that could otherwise run, would be blocked too.
Stackless tasklets that "want to run" ought to be running due to events transpiring. One kind of event is "time passes" (reactor.callLater), other types of events are user interaction (like callbacks in a UI such as GTK) or I/O events (dataReceived, connectionLost, et. al.)
There should be no need for native threads in a Twisted+Stackless universe. Even if all your tasklets constantly want to be doing work, they should be able to do that via a tasklet scheduler that is always running with a 0 callLater timeout.
Thanks for putting it so succinctly. That's what I've been trying to tell people :-) (in other media). -- Christopher Armstrong International Man of Twistery http://radix.twistedmatrix.com/ http://twistedmatrix.com/ http://canonical.com/
participants (3)
-
Andrew Francis -
Christopher Armstrong -
glyph@divmod.com