[Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).

[Gustavo Carneiro]

On 08/12/2007, Guido van Rossum <guido at python.org> wrote:
>
> On Dec 8, 2007 3:58 AM, Gustavo Carneiro <gjcarneiro at gmail.com> wrote:
> > Not only that, but pygtk is a generic module;
>
> What does "generic" mean in this context? Surely not that it applies
> to other libraries than GTK?


Well, actually this is also a PyGObject issue, not only PyGtk.  PyGObject
wraps GLib.  GLib was created to serve the needs of Gtk+, but is useful by
itself for writing portable programs.  Among other things, GLib offers a
generic "main loop", which programs can use to do generic event-driven
programming, such as timeouts, polling file descriptors, and doing other
work when the main loop becomes idle.

You could argue that non-gtk programs are rare and we shouldn't worry too
much about them.  Maybe it's true, I don't know.

> who are we to forbid the usage
> > of signals if python itself allows it?  If we were to ignore signals
> then
> > sooner or later someone would come along and shout "hey, signals work
> just
> > fine in pure python, so why did pygtk have to break my signals?".
>
> Um, signals don't "work just fine" in pure Python. And I would argue
> they don't either in C. They are extremely subtle, and most code using
> signals is broken in some way. Just try to read the sigaction() man
> page and claim you understand it.
>
> Unfortunately, in Unix there are some conditions that can only be
> delivered using signals (e.g. SIGINTR, SIGTERM) and others for which
> your choices are either polling or signals (SIGCHILD, SIGWINCH).
> Traditionally, solutions based on select() or poll() with a short
> timeout (e.g. 20 or 100 ms) have worked well, as the number of
> instructions executed each time is really negligeable, and the
> response time is still reasonable on a human time scale. Unfortunately
> it seems recent developments in power management for ultra-low power
> devices have made this an issue again.
>
> Windows solves this more elegantly by having a unified "wait for
> multiple conditions" system call which can wait on I/O, semaphores,
> and other events (within the same process or coming from other
> processes). Unfortunately, in Unix, some events don't have a file
> descriptor associated with them, and for those select()/poll() cannot
> be used.
>
> The best solution I can think of is to add a new API that takes a
> signal and a file descriptor and registers a C-level handler for that
> signal which writes a byte to the file descriptor. You can then create
> a pipe, connect the signal handler to the write end, and add the read
> end to your list of file descriptors passed to select() or poll(). The
> handler must be written in C in order to avoid the race condition
> referred to by Glyph (signals arriving after the signal check in the
> VM main loop but before the select()/poll() system call is entered
> will not be noticed until the select()/poll() call completes).


Funny that everyone mentions this solution, as it is the solution
implemented by my patch :-)

Well, to be fair, it might not be _exactly_ what is implemented by the
patch.  Reading between the lines, I think what you mean is to have python's
C signal handler mostly untouched; it would only write a byte to a pipe _in
addition to_ the normal setting the flag and Py_AddPendingCall.

The patch I submitted, on the other hand, completely removes the vector of
flags and Py_AddPendingCall, and instead writes the number of the signal
that was raised into the pipe, and reads it back from the pipe in the Python
main loop.

Which is the best solution?  I think my patch fixes two problems: 1. the
need to have a FD to wake up poll() (t o fix the problem with what we are
discussing in this thread), and 2. make Python's signal handling more
reliable (not 100% reliable because it doesn't handle longer bursts of
signals than the pipe buffer can take, but at least is race free).

My solution is being reject because people are afraid to touch the signal
handling code, which has its faults, but well know faults.  But if I
refactor the patch to keep the crappy-but-sort-of-working signal code, and
only enhance it with the pipe, maybe it will more likely get accepted.

Do I understand correctly? :-)

-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071208/66fe9718/attachment.htm