[Twisted-Python] Foreign event loop friendly reactor
In response to the "wxpython issues" thread I had suggested that we should have a thread-using reactor that integrates with foreign event loops, so I wrote one and committed it (twisted.internet.threadedselectreactor). Since this approach requires less maintenance, and whomever maintains the select reactor can trivially maintain this as well, I'm "officially" deprecating cfreactor (although I never really cared much to maintain it anyway).
An example of using it to integrate with a foreign event loop is at: doc/core/examples/Cocoa/SimpleWebClient
It should be relatively trivial to show how it could be used to integrate with other event loops (wx, Tkinter, pygame, etc.). Effectively, it's compatible with anything that has a thread-safe way to notify the main event loop.
In the future, it might make sense to revise the reactor API to support "interleave" directly, and change the existing reactors accordingly so that more than just select can support this behavior.
-bob
Bob Ippolito wrote:
In response to the "wxpython issues" thread I had suggested that we should have a thread-using reactor that integrates with foreign event loops, so I wrote one and committed it (twisted.internet.threadedselectreactor).
I wonder if anyone could be tempted into whipping up a minimal wx example with this new reactor? We've got 5 days left to finish a mountain of code for a conference demo in San Francisco - a robust wx/twisted app is just part of this. We're currently spawning a thread from within wx. It works ... but it sounds like we'd be better off with things reversed as above.
Cheers, Darran.
On Apr 17, 2005, at 6:11 AM, Darran Edmundson wrote:
Bob Ippolito wrote:
In response to the "wxpython issues" thread I had suggested that we should have a thread-using reactor that integrates with foreign event loops, so I wrote one and committed it (twisted.internet.threadedselectreactor).
I wonder if anyone could be tempted into whipping up a minimal wx example with this new reactor? We've got 5 days left to finish a mountain of code for a conference demo in San Francisco - a robust wx/twisted app is just part of this. We're currently spawning a thread from within wx. It works ... but it sounds like we'd be better off with things reversed as above.
Yeah, done. Check doc/core/examples/threadedselect (I moved the Cocoa example in there as well)
-bob
On Apr 17, 2005, at 5:42 AM, Bob Ippolito wrote:
In response to the "wxpython issues" thread I had suggested that we should have a thread-using reactor that integrates with foreign event loops, so I wrote one and committed it (twisted.internet.threadedselectreactor). Since this approach requires less maintenance, and whomever maintains the select reactor can trivially maintain this as well, I'm "officially" deprecating cfreactor (although I never really cared much to maintain it anyway).
Seems like a good idea. Does it actually fix WX? That is, does wxCallAfter not suffer from the "oh you wanted it to happen *now*, not a second later" problem?
James
On Apr 17, 2005, at 12:02 PM, James Y Knight wrote:
On Apr 17, 2005, at 5:42 AM, Bob Ippolito wrote:
In response to the "wxpython issues" thread I had suggested that we should have a thread-using reactor that integrates with foreign event loops, so I wrote one and committed it (twisted.internet.threadedselectreactor). Since this approach requires less maintenance, and whomever maintains the select reactor can trivially maintain this as well, I'm "officially" deprecating cfreactor (although I never really cared much to maintain it anyway).
Seems like a good idea. Does it actually fix WX? That is, does wxCallAfter not suffer from the "oh you wanted it to happen *now*, not a second later" problem?
Try it.
-bob
Thanks for your work in developing this new reactor. I just have a couple questions:
Is it (thread) safe to call wx methods from within call/errbacks with this reactor? I ask because I was (based upon the example at http://solipsis.netofpeers.net/wiki/wikka.php?wakka=WxTwistedExample) translating callbacks into wx.Events executed during the next wx loop (the PostEvent function is thread safe). If I am understanding the threadedselectreactor.py code correctly it is "waking" the wx loop and therefore this should not be needed since only one mainloop will be active at a time.
Also my understanding is that the reactor.interleave function should be called ONCE in order to provide a function to wake the foreign event loop. I am doing this in MyApp.OnInit() and it seems to work fine. The example shows this being executed in the frame's __init__ function and I do not think this will work for multiple frames (I'm not saying there is anything WRONG with the example, I'm just trying to make sure I understand the correct use of the new reactor when dealing with multiple wxWindow objects).
I did not have any problems converting my test application to use the new reactor, but I wanted to clarify these items so they don't bite me later on.
Thanks once again,
Shawn
sl_church@sbcglobal.net http://SChurchComputers.com
Bob Ippolito wrote:
In response to the "wxpython issues" thread I had suggested that we should have a thread-using reactor that integrates with foreign event loops, so I wrote one and committed it (twisted.internet.threadedselectreactor). Since this approach requires less maintenance, and whomever maintains the select reactor can trivially maintain this as well, I'm "officially" deprecating cfreactor (although I never really cared much to maintain it anyway).
An example of using it to integrate with a foreign event loop is at: doc/core/examples/Cocoa/SimpleWebClient
It should be relatively trivial to show how it could be used to integrate with other event loops (wx, Tkinter, pygame, etc.). Effectively, it's compatible with anything that has a thread-safe way to notify the main event loop.
In the future, it might make sense to revise the reactor API to support "interleave" directly, and change the existing reactors accordingly so that more than just select can support this behavior.
-bob
Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On Apr 17, 2005, at 6:21 PM, Shawn Church wrote:
Bob Ippolito wrote:
In response to the "wxpython issues" thread I had suggested that we should have a thread-using reactor that integrates with foreign event loops, so I wrote one and committed it (twisted.internet.threadedselectreactor). Since this approach requires less maintenance, and whomever maintains the select reactor can trivially maintain this as well, I'm "officially" deprecating cfreactor (although I never really cared much to maintain it anyway).
An example of using it to integrate with a foreign event loop is at: doc/core/examples/Cocoa/SimpleWebClient
It should be relatively trivial to show how it could be used to integrate with other event loops (wx, Tkinter, pygame, etc.). Effectively, it's compatible with anything that has a thread-safe way to notify the main event loop.
In the future, it might make sense to revise the reactor API to support "interleave" directly, and change the existing reactors accordingly so that more than just select can support this behavior.
Thanks for your work in developing this new reactor. I just have a couple questions:
Is it (thread) safe to call wx methods from within call/errbacks with this reactor? I ask because I was (based upon the example at http://solipsis.netofpeers.net/wiki/wikka.php?wakka=WxTwistedExample) translating callbacks into wx.Events executed during the next wx loop (the PostEvent function is thread safe).
Forget everything you ever learned about wxPython/Twisted integration. It is no longer relevant. Forget that any of the ugly wxPython+Twisted Python Cookbook entries, lame wiki examples, twisted.internet.wxreactor, etc. even exist.
If I am understanding the threadedselectreactor.py code correctly it is "waking" the wx loop and therefore this should not be needed since only one mainloop will be active at a time.
threadedselectreactor lets you imagine that you're in a wonderful single threaded world where everything happens in a context where calling into Twisted and wxPython are both safe (unless you explicitly make life suck by creating more threads).
Also my understanding is that the reactor.interleave function should be called ONCE in order to provide a function to wake the foreign event loop. I am doing this in MyApp.OnInit() and it seems to work fine. The example shows this being executed in the frame's __init__ function and I do not think this will work for multiple frames (I'm not saying there is anything WRONG with the example, I'm just trying to make sure I understand the correct use of the new reactor when dealing with multiple wxWindow objects).
Yes it should be called exactly once. The example does it for the main frame. It is done there because that is the same frame that quits the application when it's closed, so it also integrates with reactor.stop.
-bob
On Sun, 2005-04-17 at 18:45 -0400, Bob Ippolito wrote:
Forget everything you ever learned about wxPython/Twisted integration. It is no longer relevant. Forget that any of the ugly wxPython+Twisted Python Cookbook entries, lame wiki examples, twisted.internet.wxreactor, etc. even exist.
We should update the GUI event loop integration howto to point at the new method, and deprecate wxsupport and wxreactor too, in that case.
On Sun, 17 Apr 2005 22:22:32 -0400, Itamar Shtull-Trauring itamar@itamarst.org wrote:
On Sun, 2005-04-17 at 18:45 -0400, Bob Ippolito wrote:
Forget everything you ever learned about wxPython/Twisted integration. It is no longer relevant. Forget that any of the ugly wxPython+Twisted Python Cookbook entries, lame wiki examples, twisted.internet.wxreactor, etc. even exist.
We should update the GUI event loop integration howto to point at the new method, and deprecate wxsupport and wxreactor too, in that case.
Why not just change wxreactor.install() to use this new code, rather than deprecating it? That would leave the door open to new strategies in case wx ever gets a non-broken eventloop.
On Apr 18, 2005, at 1:19 AM, glyph@divmod.com wrote:
On Sun, 17 Apr 2005 22:22:32 -0400, Itamar Shtull-Trauring itamar@itamarst.org wrote:
On Sun, 2005-04-17 at 18:45 -0400, Bob Ippolito wrote:
Forget everything you ever learned about wxPython/Twisted integration. It is no longer relevant. Forget that any of the ugly wxPython+Twisted Python Cookbook entries, lame wiki examples, twisted.internet.wxreactor, etc. even exist.
We should update the GUI event loop integration howto to point at the new method, and deprecate wxsupport and wxreactor too, in that case.
Why not just change wxreactor.install() to use this new code, rather than deprecating it? That would leave the door open to new strategies in case wx ever gets a non-broken eventloop.
I look at it this way:
- Using a thread for a socket event source isn't a bad idea (it's what Qt/Cocoa/etc. are doing behind your back anyway), and can give us options for even better performance on multi-core/multi-processor machines when a foreign event loop is in control (i.e. reads and writes could also happen in the thread).
- Writing wrappers for the socket/timer stuff in GUI frameworks sucks, because they're often too different, limited, or buggy
- The implementations of socket wrappers and timers in GUI frameworks are usually slow, incomplete, buggy, etc. (again).
- Not caring about GUI reactors lets you deprecate > 1300 lines of reactor code in exchange for a couple examples and a short support library or two that we should have anyway
- The reactor interface could be extended such that threadedselectreactor goes away, and the select/kqueue/poll/epoll/iocp reactors become compatible with interleave(), which lets all reactors be compatible with foreign event loops while still maintaining the choice to do whatever is teh smart for your platform.
-bob
Bob Ippolito wrote:
On Apr 18, 2005, at 1:19 AM, glyph@divmod.com wrote:
Why not just change wxreactor.install() to use this new code, rather than deprecating it? That would leave the door open to new strategies in case wx ever gets a non-broken eventloop.
I look at it this way:
I don't disagree with any of your points, I'm just saying that we don't need to deprecate an interface every time we change an implementation. *some* event loops (e.g. gtk) are actually better than the normal reactor one, so it still makes sense to choose on that basis in some cases. In the cases where the threadedselectreactor is better-performing than the direct-GUI-integration approach, that should be a decision that the Twisted core is aware of, but not necessarily one that the application developer need be made aware of.
participants (7)
-
Bob Ippolito -
Darran Edmundson -
Glyph Lefkowitz -
glyph＠divmod.com
-
Itamar Shtull-Trauring -
James Y Knight -
Shawn Church