[Python-Dev] Signals, threads, blocking C functions

[Nick Maclaren]

I was hoping to have stopped, but here are a few comments.

I agree with Jan Kanis.  That is the way to tackle this one.

"Adam Olsen" <rhamph at gmail.com> wrote:
>         
> I don't think we should let this die, at least not yet.  Nick seems to
> be arguing that ANY signal handler is prone to random crashes or
> corruption (due to bugs).  However, we already have a signal handler,
> so we should already be exposed to the random crashes/corruption.

No.  I am afraid that is a common myth and often catastrophic mistake.
In this sort of area, NEVER assume that even apparently unrelated changes
won't cause 'working' code to misbehave.  Yes, Python is already exposed,
but it would be easy to turn a very rare failure into a more common one.

What I was actually arguing for was defensive programming.

> If we're going to rely on signal handling being correct then I think
> we should also rely on write() being correct.  Note that I'm not
> suggesting an API that allows arbitrary signal handlers, but rather
> one that calls write() on an array of prepared file descriptors
> (ignoring errors).

For your interpretation of 'correct'.  The cause of this chaos is that
the C and POSIX standards are inconsistent, even internally, and they
are wildly incompatible.  So, even if things 'work' today, don't bet on
the next release of your favourite system behaving the same way.

It wouldn't matter if there was a de facto standard (i.e. a consensus),
but there isn't.

> Ensuring modifications to that array are atomic would be tricky, but I
> think it would be doable if we use a read-copy-update approach (with
> two alternating signal handler functions).  Not sure how to ensure
> there's no currently running signal handlers in another thread though.
>  Maybe have to rip the atomic read/write stuff out of the Linux
> sources to ensure it's *always* defined behavior.

Yes.  But even that wouldn't solve the problem, as that code is very
gcc-specific.

> Looking into the existing signalmodule.c, I see no attempts to ensure
> atomic access to the Handlers data structure.  Is the current code
> broken, at least on non-x86 platforms?

Well, at a quick glance at the actual handler (the riskiest bit):

    1) It doesn't check the signal range - bad practice, as systems
do sometimes generate wayward numbers.

    2) Handlers[sig_num].tripped = 1; is formally undefined, but
actually pretty safe.  If that breaks, nothing much will work.  It
would be better to make the int sig_atomic_t, as you say.

    3) is_tripped++; and Py_AddPendingCall(checksignals_witharg, NULL);
will work only because the handler ignores all signals in subthreads
(which is definitely NOT right, as the comments say).

Despite the implication, the code of Py_AddPendingCall is NOT safe
against simultaneous registration.  It is just plain broken, I am
afraid.  The note starting "Darn" should be a LOT stronger :-)

[ For example, think of two threads calling the function at exactly
the same time, in almost perfect step.  Oops. ]

I can't honestly promise to put any time into this in the forseeable
future, but will try (sometime).  If anyone wants to tackle this,
please ask me for comments/help/etc.


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1 at cam.ac.uk
Tel.:  +44 1223 334761    Fax:  +44 1223 334679