[Twisted-Python] Gtk2Reactor: what are the reasons for avoiding GLib functions?
I'd like to try and get to the bottom of the various problems with Twisted + GTK + Windows (mainly issue #4932[1], but see also issues #4376[2], #4862[3], #1759[4]). On issue #4376 there's a comment from Exarkun: "glib handles timeouts much less efficiently than any of the existing Twisted reactors. Letting glib take care of these would probably make the glib-based reactors unusable for applications with many timeouts." Is there a benchmark or piece of code somewhere that verifies this (or at least gives some sort of demonstration of "unuseable")? If not, is there some alternative implementation lying around in version control that tried to use GLib's mainloop functions and was thrown out for performance reasons? I ask because my first approach to fixing this would be to ignore what's already there and try to write something that's quite closely coupled to the GLib/GTK mainloop. But it sounds like this was already tried and abandoned, and I want to be able to make sure I'm not (a) writing something that was already written, or (b) performs too poorly to be of any use. It would be good to have a metric available. Cheers, Jason [1] http://twistedmatrix.com/trac/ticket/4932 [2] http://twistedmatrix.com/trac/ticket/4376 [3] http://twistedmatrix.com/trac/ticket/4862 [4] http://twistedmatrix.com/trac/ticket/1759
Jason Heeris wrote: […]
"glib handles timeouts much less efficiently than any of the existing Twisted reactors. Letting glib take care of these would probably make the glib-based reactors unusable for applications with many timeouts."
Is there a benchmark or piece of code somewhere that verifies this (or at least gives some sort of demonstration of "unuseable")?
…and if someone does have such a benchmark, it'd be great to add it to <http://speed.twistedmatrix.com/> :) -Andrew.
On Tue, 2011-04-05 at 15:08 +0800, Jason Heeris wrote:
I'd like to try and get to the bottom of the various problems with Twisted + GTK + Windows (mainly issue #4932[1], but see also issues #4376[2], #4862[3], #1759[4]).
Thanks for looking into this!
On issue #4376 there's a comment from Exarkun:
"glib handles timeouts much less efficiently than any of the existing Twisted reactors. Letting glib take care of these would probably make the glib-based reactors unusable for applications with many timeouts."
Is there a benchmark or piece of code somewhere that verifies this (or at least gives some sort of demonstration of "unuseable")?
https://bugzilla.gnome.org/show_bug.cgi?id=143061 I don't think it's ever going to get fixed on the gtk side, I filed that bug in 2004. The issues on Windows may be different than this, for that matter.
Hello, On 5.4.2011 9:08, Jason Heeris wrote:
I'd like to try and get to the bottom of the various problems with Twisted + GTK + Windows (mainly issue #4932[1], but see also issues #4376[2], #4862[3], #1759[4]).
On issue #4376 there's a comment from Exarkun:
"glib handles timeouts much less efficiently than any of the existing Twisted reactors. Letting glib take care of these would probably make the glib-based reactors unusable for applications with many timeouts."
Is there a benchmark or piece of code somewhere that verifies this (or at least gives some sort of demonstration of "unuseable")?
I think that the main reason for PortableGtkReactor being used on Windows is due to glib's IO channels limitations on Windows. You can only have 63 channels in the event loop at the same time due to g_poll's use of MsgWaitForMultipleObject. In addition, IO channels are quite buggy on Windows when used for sockets, see the comment at the top of the glib/giowin32.c file in the glib source for a list of issues. E.g. I suspect that the second problem described in: https://bugzilla.gnome.org/show_bug.cgi?id=357674 is responsible for Twisted's ticket #3371, because a similar problem can be seen with win32eventreactor (ticket #4950), which also uses WSAEventSelect for socket monitoring. Best regards, Ziga
On 5 April 2011 20:06, Žiga Seilnacht <ziga.seilnacht@gmail.com> wrote:
I think that the main reason for PortableGtkReactor being used on Windows is due to glib's IO channels limitations on Windows. You can only have 63 channels in the event loop at the same time due to g_poll's use of MsgWaitForMultipleObject.
The trouble is, PortableGtkReactor is just out-and-out unusable for certain applications. In my case, it turned a 3 minute comms session into a 25 hour session — OR, I could have my app completely max out the CPU while it's running. Neither is workable in any realistic context. But I'm pretty sure I don't need > 63 I/O channels open at once, unless I'm missing something about the inner workings of Twisted. (I mean no offense, I can see why it has to work the way it does... but its weakness shows up in exactly the kind of protocol I need to use.)
In addition, IO channels are quite buggy on Windows when used for sockets, see the comment at the top of the glib/giowin32.c file in the glib source for a list of issues. E.g. I suspect that the second problem described in:
https://bugzilla.gnome.org/show_bug.cgi?id=357674
is responsible for Twisted's ticket #3371, because a similar problem can be seen with win32eventreactor (ticket #4950), which also uses WSAEventSelect for socket monitoring.
Hmm. I may be in over my head here. But I won't give up just yet, since it *would* be nice to be able to use PyGTK for the things I need to do. Thanks for the info, Jason
participants (4)
-
Andrew Bennetts -
Itamar Turner-Trauring -
Jason Heeris -
Žiga Seilnacht