[Python-Dev] Signals, threads, blocking C functions

[Nick Maclaren]

"Gustavo Carneiro" <gjcarneiro at gmail.com> wrote:
> 
> We have to resort to timeouts in pygtk in order to catch unix signals
> in threaded mode.

A common defect of modern designs - TCP/IP is particularly objectionable
in this respect, but that battle was lost and won over two decades ago :-(

> The reason is this.  We call gtk_main() (mainloop function) which
> blocks forever.  Suppose there are threads in the program; then any
> thread can receive a signal (e.g. SIGINT).  Python catches the signal,
> but doesn't do anything; it simply sets a flag in a global structure
> and calls Py_AddPendingCall(), and I guess it expects someone to call
> Py_MakePendingCalls().  However, the main thread is blocked calling a
> C function and has no way of being notified it needs to give control
> back to python to handle the signal.  Hence, we use a 100ms timeout
> for polling.  Unfortunately, timeouts needlessly consume CPU time and
> drain laptop batteries.

Yup.

> According to [1], all python needs to do to avoid this problem is
> block all signals in all but the main thread; then we can guarantee
> signal handlers are always called from the main thread, and pygtk
> doesn't need a timeout.

1) That page is password protected, so I can't see what it says, and
am disinclined to register myself to yet another such site.

2) No way, Jose, anyway.  The POSIX signal handling model was broken
beyond redemption, even before threading was added, and the combination
is evil almost beyond belief.  That procedure is good practice, yes,
but that is NOT all that you have to do - it may be all that you CAN
do, but that is not the same.

Come back MVS (or even VMS) - all is forgiven!  That is only partly
a joke.

> Another alternative would be to add a new API like
> Py_AddPendingCallNotification, which would let python notify
> extensions that new pending calls exist and need to be processed.

Nope.  Sorry, but you can't solve a broken design by adding interfaces.

>   But I would really prefer the first alternative, as it could be
> fixed within python 2.5; no need to wait for 2.6.

It clearly should be done, assuming that Python's model is that it
doesn't want to get involved with subthread signalling (and I really,
but REALLY, recommend not doing so).  The best that can be done is to
say that all signal handling is the business of the main thread and
that, when the system bypasses that, all bets are off.

>   Please, let's make Python ready for the enterprise! [2]

Given that no Unix variant or Microsoft system is, isn't that rather
an unreasonable demand?

I am probably one of the last half-dozen people still employed in a
technical capacity who has implemented run-time systems that supported
user-level signal handling with threads/asynchronicity and allowing
for signals received while in system calls.  It would be possible to
modify/extend POSIX or Microsoft designs to support this, but currently
they don't make it possible.  There is NOTHING that Python can do but
to minimise the chaos.


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1 at cam.ac.uk
Tel.:  +44 1223 334761    Fax:  +44 1223 334679