Running embedded python shell on a separate thread

[Roel van de Kraats]

For those who are interested,

In an application I am working on, an embedded python shell needs to be 
run on a separate thread, since the main thread is already 'taken' (by 
Qt). This causes an issue with handling SIGINT (Control-C) when using 
the readline package; the 'KeyboardInterrupt' is only shown/handled 
after pressing <ENTER>.

With a work-around, I was able to get this working 'normally' without 
modifying the python code itself, which I want to share with you.

For completeness, I'm using Python 2.6.5 on Ubuntu Lucid. Our 
application is in C++.

The problem is that Control-C causes a SIGINT signal to be sent to the 
main/initial thread. Python's signal handler (signal_handler in 
Modules/signalmodule.c) handles this signal by marking that it occurred 
and adding a 'pending call'. But the readline functionality 
(readline_until_enter_or_signal in Modules/readline.c) only notices this 
when select() returns errno==EINTR. That is, when select() itself was 
interrupted. And that only happens when the SIGINT signal is sent to the 
thread on which readline is running.

The work-around I made was to install my own SIGINT signal handler with 
this functionality:
- When it receives a signal on a thread different from the 'python' 
thread, it uses pthread_kill() to forward the signal to that 'python' 
thread.
- When it receives a signal on the 'python' thread, it temporarily 
restores the 'python' signal handler and raises the signal again.

Installing the signal handler is done by a wrapper around the readline 
function, through the PyOS_ReadlineFunctionPointer hook.

My application code (simplified C++) can be found below.

I guess all these 'hacks' could be prevented when Python's own signal 
handler would forward the SIGINT signal (or any signal) to the 'python' 
thread.

Kind regards,
     Roel van de Kraats


typedef void (*sighandler_t)(int);

// don't use PyOS_setsig() since we need the SA_NODEFER flag
int set_sighandler(int sig, sighandler_t handler, sighandler_t * old_handler = NULL, int flags = SA_NODEFER)
{
     struct sigaction action;
     struct sigaction oldaction;
     memset(&action, 0, sizeof(action));
     memset(&oldaction, 0, sizeof(oldaction));
     action.sa_handler = handler;
     action.sa_flags = flags;
     int ret = sigaction(sig,&action,&oldaction);
     if (old_handler) {
         *old_handler = oldaction.sa_handler;
     }
     return ret;
}

sighandler_t old_int_signal_handler;

void int_signal_handler(int)
{
     if ( this_is_the_python_thread() ) {
         static volatile bool calling = false;   // for error checking
         if (calling) {
             // unexpected recursive call
             return;
         }
         calling = true;
         sighandler_t prev_handler;
         // restore original handler
         (void)set_sighandler(SIGINT, old_int_signal_handler,&prev_handler);
         if (prev_handler != int_signal_handler) {
             // expected int_signal_handler
             return;
         }
         // raise signal which will be handled by the original handler
         ::pthread_kill(_python_thread_id, SIGINT);
         // install our handler again
         (void)set_sighandler(SIGINT, int_signal_handler);
         calling = false;
     } else {
         // not meant for this thread; redirect to python thread
         ::pthread_kill(_python_thread_id, SIGINT);
     }
}

char *(*old_readline)(FILE *, FILE *, char *);

char * my_readline(FILE * a1, FILE * a2, char * a3)
{
     // replace signal handler
     (void)set_sighandler(SIGINT, int_signal_handler,&old_int_signal_handler);
     char * ret = (*old_readline)(a1, a2, a3);
     // restore original handler
     (void)set_sighandler(SIGINT, old_int_signal_handler, NULL, 0);
     return ret;
}

void python_thread_function()
{
     Py_Initialize();

     // some other initialization stuff here

     // make sure readline is initialized before we replace the readline function
     PyRun_SimpleString("import readline\n");

     old_readline = PyOS_ReadlineFunctionPointer;
     if (old_readline) {
         PyOS_ReadlineFunctionPointer = my_readline;
     } else {
         // PyOS_ReadlineFunctionPointer was not set so don't replace
     }

     // the python prompt appears now...
     int ret = PyRun_InteractiveLoop(stdin, "<stdin>");

     Py_Exit(ret);
}