[Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
gjcarneiro at gmail.com
Sat Dec 8 21:38:30 CET 2007
On 08/12/2007, Guido van Rossum <guido at python.org> wrote:
> On Dec 8, 2007 9:57 AM, Gustavo Carneiro <gjcarneiro at gmail.com> wrote:
> > On 08/12/2007, Guido van Rossum <guido at python.org> wrote:
> > > On Dec 8, 2007 3:58 AM, Gustavo Carneiro <gjcarneiro at gmail.com> wrote:
> > > > Not only that, but pygtk is a generic module;
> > >
> > > What does "generic" mean in this context? Surely not that it applies
> > > to other libraries than GTK?
> > Well, actually this is also a PyGObject issue, not only
> PyGtk. PyGObject
> > wraps GLib. GLib was created to serve the needs of Gtk+, but is useful
> > itself for writing portable programs. Among other things, GLib offers a
> > generic "main loop", which programs can use to do generic event-driven
> > programming, such as timeouts, polling file descriptors, and doing other
> > work when the main loop becomes idle.
> > You could argue that non-gtk programs are rare and we shouldn't worry
> > much about them. Maybe it's true, I don't know.
> > > > who are we to forbid the usage
> > > > of signals if python itself allows it? If we were to ignore signals
> > then
> > > > sooner or later someone would come along and shout "hey, signals
> > just
> > > > fine in pure python, so why did pygtk have to break my signals?".
> > >
> > > Um, signals don't "work just fine" in pure Python. And I would argue
> > > they don't either in C. They are extremely subtle, and most code using
> > > signals is broken in some way. Just try to read the sigaction() man
> > > page and claim you understand it.
> > >
> > > Unfortunately, in Unix there are some conditions that can only be
> > > delivered using signals (e.g. SIGINTR, SIGTERM) and others for which
> > > your choices are either polling or signals (SIGCHILD, SIGWINCH).
> > > Traditionally, solutions based on select() or poll() with a short
> > > timeout (e.g. 20 or 100 ms) have worked well, as the number of
> > > instructions executed each time is really negligeable, and the
> > > response time is still reasonable on a human time scale. Unfortunately
> > > it seems recent developments in power management for ultra-low power
> > > devices have made this an issue again.
> > >
> > > Windows solves this more elegantly by having a unified "wait for
> > > multiple conditions" system call which can wait on I/O, semaphores,
> > > and other events (within the same process or coming from other
> > > processes). Unfortunately, in Unix, some events don't have a file
> > > descriptor associated with them, and for those select()/poll() cannot
> > > be used.
> > >
> > > The best solution I can think of is to add a new API that takes a
> > > signal and a file descriptor and registers a C-level handler for that
> > > signal which writes a byte to the file descriptor. You can then create
> > > a pipe, connect the signal handler to the write end, and add the read
> > > end to your list of file descriptors passed to select() or poll(). The
> > > handler must be written in C in order to avoid the race condition
> > > referred to by Glyph (signals arriving after the signal check in the
> > > VM main loop but before the select()/poll() system call is entered
> > > will not be noticed until the select()/poll() call completes).
> > Funny that everyone mentions this solution, as it is the solution
> > implemented by my patch :-)
> Mind pointing me to it again? I can't find it in the Python bug tracker.
> > Well, to be fair, it might not be _exactly_ what is implemented by the
> > patch. Reading between the lines, I think what you mean is to have
> > C signal handler mostly untouched; it would only write a byte to a pipe
> > addition to_ the normal setting the flag and Py_AddPendingCall.
> Well, in many cases I see no problems with the current signal handler,
> and people are used to it, so I'm not sure what is gained by getting
> rid of it.
> > The patch I submitted, on the other hand, completely removes the vector
> > flags and Py_AddPendingCall, and instead writes the number of the signal
> > that was raised into the pipe, and reads it back from the pipe in the
> > main loop.
> I believe Py_AddPendingCall was meant to be used by other places
> besides the signal handling.
True. The patch does not remove Py_AddPendingCall, only stops using it for
> Which is the best solution? I think my patch fixes two problems: 1. the
> > need to have a FD to wake up poll() (t o fix the problem with what we
> > discussing in this thread), and 2. make Python's signal handling more
> > reliable (not 100% reliable because it doesn't handle longer bursts of
> > signals than the pipe buffer can take, but at least is race free).
> I think it's okay to drop signals if too many come. The FD should be
> put in non-blocking mode so the signal handler won't block forever.
> Does Unix even promise that a signal gets delivered twice if it gets
> sent quickly twice in a row?
> My solution is being reject because people are afraid to touch the signal
> > handling code, which has its faults, but well know faults. But if I
> > refactor the patch to keep the crappy-but-sort-of-working signal code,
> > only enhance it with the pipe, maybe it will more likely get accepted.
> > Do I understand correctly? :-)
> Not really; I don't recall seeing your patch. And I object to your
> subjective description of the existing signal handling; it has served
> us well enough.
Sorry. I have to say existing code is very bad in order to convince people
it is worth changing... :P
Anyway, the approach of writing to a pipe and reading it back later appears
(to me) simpler to read and understand. That should also count for
I have a worry though -- if signal handlers *always* and *only*
> communicate by writing a byte to a FD, does that mean that the VM main
> loop will have to attempt to read a byte from a pipe every time it
> checks for signals? That sounds very expensive for something that's
> not needed by the vast majority of Python applications.
Yes, it would be expensive. Instead the patch uses a simple global boolean
value, set to true when a signal is received, after writing to the pipe.
PyErr_CheckSignals() does not try to read from the pipe unless that value is
Plus, you will
> have to expose the FD to the user so that it can be included in
> select() and poll() calls.
Anyway, let's see the patch first.
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Python-Dev