[Numpy-discussion] Handling interrupts in NumPy extensions

[David M. Cooke]

On Wed, 23 Aug 2006 11:45:29 -0700
Travis Oliphant <oliphant.travis at ieee.org> wrote:

> 
> I'm working on some macros that will allow extensions to be 
> "interruptable" (i.e. with Ctrl-C).  The idea came from SAGE but the 
> implementation is complicated by the possibility of threads and making 
> sure to handle clean-up code correctly when the interrupt returns.
> 

For writing clean-up code, here's some prior art on adding exceptions to C:

http://www.ossp.org/pkg/lib/ex/  (BSD license)
http://adomas.org/excc/    (GPL'd, so no good)
http://ldeniau.web.cern.ch/ldeniau/html/exception/exception.html  (no license
given)

The last one has functions that allow you to add pointers (and their
deallocation functions) to a list so that they can be deallocated when an
exception is thrown.

(You don't necessarily need something like these libraries, but I thought I'd
throw it in here, because it's along the same lines)

> Step 2:
> 
> Implementation.  I have the idea to have a single interrupt handler 
> (defined globally in NumPy) that basically uses longjmp to return to the 
> section of code corresponding to the thread that is handling the 
> interrupt.  I had thought to use a global variable containing a linked 
> list of jmp_buf structures with a thread-id attached 
> (PyThread_get_thread_ident()) so that the interrupt handler can search 
> it to see if the thread has registered a return location.  If it has 
> not, then the intterupt handler will just return normally.   In this way 
> a thread that calls setjmpbuf will be sure to return to the correct 
> place when it handles the interrupt.

Signals and threads don't mix well at *all*. With POSIX semantics, synchronous
signals (ones caused by the thread itself) should be sent to the handler for
that thread. Asynchronous ones (like SIGINT for Ctrl-C) will be sent to an
*arbitrary* thread. (Apple, for instance, doesn't make any guarantees on
which thread gets it: http://developer.apple.com/qa/qa2001/qa1184.html)

Best way I can see this is to have a SIGINT handler installed that sets a
global variable, and check that every so often. It's such a good way that
Python already does this -- Parser/intrcheck.c sets the handler, and you can
use PyOS_InterruptOccurred() to check if one happened. So something like

while (long running loop) {
   if (PyOS_InterruptOccurred()) goto error:
   ... useful stuff ...
}
error:

This could be abstracted to a set of macros (with Perry's syntax):

NPY_SIG_INTERRUPTABLE
  while (long loop) {
     NPY_CHECK_SIGINT;
     .. more stuff ..
  }
NPY_SIG_END_INTERRUPTABLE

where NPY_CHECK_SIGINT would do a longjmp().

Or come up with a good (fast) way to run stuff in another process :-)

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                      http://arbutus.physics.mcmaster.ca/dmc/
|cookedm at physics.mcmaster.ca