[Numpy-discussion] Handling interrupts in NumPy extensions

[David Cournapeau]

Travis Oliphant wrote:
> David Cournapeau wrote:
>> Indeed.
>>
>> By the way, I tried something for python.thread + signals. This is posix 
>> specific, and it works as expected on linux:
>>   
> Am I right that this could this be accomplished simply by throwing away 
> all the interrupt handling stuff in the code and checking for 
> PyOS_InterruptOccurred() in the place where you check for the global 
> variable that your signal handler uses?  Your signal handler does 
> essentially what Python's signal handler already does, if I'm not mistaken.
I don't know how the python signal handler works, but I believe it should do
more or less the same, indeed. The key idea is that it is important to mask
other signals related to interrupting. To have a relatively clear view 
on this, if you have not seen it, you may take a look at the gnu C doc 
on signal handling:

http://www.gnu.org/software/libc/manual/html_node/Defining-Handlers.html#Defining-Handlers

After having given some thought, I am wondering about what exactly we 
are trying to do:
    - the main problem is to be able to interrupt some which may take a 
long time to compute, without corrupting the whole python process.
    - for that, those function need to be able to trap the usual signals 
corresponding to interrupt (SIGINT, etc... on Unix, equivalents on windows).

There are two ways to handle a signal:
    - check regularly some global (that is, global to the whole process) 
value, and if change this value if a signal is trapped. That's the 
easier way, but this is not thread safe as I first thought (I will code 
an example if I have time).
    - the signal handler jumps to an other point of the program where 
cleaning is done: this is more complicated, and I am not sure we need 
the complication (I have never used this scheme, so I may just miss the 
point totally). I don't even want to think how it works in 
multi-threading environment :)

Now, the threading issue came in, and I am not sure why we need to care: 
this is a problem if numpy is implemented in a multi-thread way, but I 
don't believe it to be the case, right ? An other solution, which is 
used I think in more sophisticated programs, is having one thread with 
high priority, which only job is to detect signals, and to mask all 
signals in all other threads. Again, this seems  overkill (and highly 
non portable) ? And this should be the python interpreter job, no ?

Actually, as this is a generic problem for any python extension code, 
other really smart people should have thought about that... If I am 
interpreting correctly what is said here 
http://docs.python.org/lib/module-signal.html, I believe that what you 
suggest (using PyOS_InterruptOccurred() at some points) is what shall be 
done: the python interpreter is making sure that the signal is send to 
the main thread, that is the thread where numpy is executed (that's my 
understanding on the way python interpreter works, not a fact).
 
David