[Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).

[Adam Olsen]

On Dec 7, 2007 2:35 PM,  <glyph at divmod.com> wrote:
>
> On 02:48 pm, gjcarneiro at gmail.com wrote:
> >Not only that, but current python signal handling is not theorethically
> >async safe; there are race conditions in the Py_AddPendingCalls API,
> >and it
> >just happens to work most of the time.

[This refers to the internal datastructures used by
Py_AddPendingCalls, which aren't updated in a safe way.
Hard/impossible to fix in C, but fairly easy with embedded assembly.]


> Twisted has encountered one such issue, described here:
>
>     http://twistedmatrix.com/trac/ticket/1997#comment:12

[This refers to the overall design, which is inherently racey.]


> Unfortunately, I don't know enough about signals to suggest or comment
> on the solution.  Any Python/C wrapper around a syscall which can be
> interrupted needs to somehow atomically check for the presence of
> pending python signal handlers; I don't know of any POSIX API to do
> that.

Overall, what you'd need to do is register a wakeup function (to be
called by a signal handler or another thread), and have that wakeup
function cancel whatever you're doing.  The hard part is it needs to
work at *ANY* time while it's registered, before you've even called
the library function or syscall you intend to cancel!

I currently know of two methods of achieving this:
1) If reading a file or socket, first poll the fd, then do a
non-blocking read.  The wakeup function writes to a wakeup pipe you
also poll, which then wakes you up.  A wakeup after poll completes is
ignored, but the non-blocking read will finish promptly enough anyway.
2) Use sigsetjmp before a syscall (I wouldn't trust a library call),
then have the signal handler jump completely out of the operation.
This is evil and unportable, but probably works.

Additionally, this only gets SIGINT with the default behaviour to work
right, as it can be entirely implemented in C.  If you want to handle
arbitrary signals running arbitrary python code you really need a
second thread to run them in.

-- 
Adam Olsen, aka Rhamphoryncus