[Python-Dev] Signals, threads, blocking C functions

[Gustavo Carneiro]

  In GLib we have a child watch notification feature that relies on
the following signal handler:

static void
g_child_watch_signal_handler (int signum)
{
  child_watch_count ++;

  if (child_watch_init_state == CHILD_WATCH_INITIALIZED_THREADED)
    {
      write (child_watch_wake_up_pipe[1], "B", 1);
    }
  else
    {
      /* We count on the signal interrupting the poll in the same thread.
       */
    }
}

  Now, we've had this API for a long time already (at least 2.5
years).  I'm pretty sure it works well enough on most *nix systems.
Event if it works 99% of the times, it's way better than *failing*
*100%* of the times, which is what happens now with Python.

  All I ask is an API to add a callback that Python signal handlers
call, from signal context.  That much I'm sure is safe.  What happens
from there on will be out of Python's hands, so Python
purist^H^H^H^H^H^H developers cannot be blamed for anything that
happens next.  You can laugh at PyGTK and GLib all you want for having
"unsafe signal handling", I don't care.

  Regards.

On 9/4/06, Nick Maclaren <nmm1 at cus.cam.ac.uk> wrote:
> Jean-Paul Calderone <exarkun at divmod.com> wrote:
> >
> > Thanks for expounding.  Given that it is basically impossible to do
> > anything useful in a signal handler according to the relevant standards
> > (does Python's current signal handler even avoid relying on undefined
> > behavior?), how would you suggest addressing this issue?
>
> Much as you are doing, and I described, but the first step would be
> to find out what 'most' Python people need for signal handling in
> threaded programs.  This is because there is an unavoidable conflict
> between portability/reliability and functionality.
>
> I would definitely block all signals in threads, except for those that
> are likely to be generated ON the thread (SIGFPE etc.)  It is a very
> good idea not to touch the handling of several of those, because doing
> so can cause chaos.
>
> I would have at least two 'standard' handlers, one of which would simply
> set a flag and return, and the other of which would abort.  Now, NEITHER
> is a very useful specification, but providing ANY information is risky,
> which is why it is critical to know what people need.
>
> I would not TRUST the blocking of signals, so would set up handlers even
> when I blocked them, and would do the minimum fiddling in the main
> thread compatible with decent functionality.
>
> I would provide a call to test if the signal flag was set, and another
> to test and clear it.  This would be callable ONLY from the main thread,
> and that would be checked.
>
> It is possible to do better, but that starts needing serious research.
>
> > It seems to me that it is actually possible to do useful things in a
> > signal handler, so long as one accepts that doing so is relying on
> > platform specific behavior.
>
> Unfortunately, that is wrong.  That was true under MVS and VMS, but
> in Unix and Microsoft systems, the problem is that the behaviour is
> both platform and circumstance-dependent.  What you can do reliably
> depends mostly on what is going on at the time.
>
> For example, on many Unix and Microsoft platforms, signals received
> while you are in the middle of certain functions or system calls, or
> certain particular signals (often SIGFPE), call the C handler with a
> bad set of global pointers or similar.  I believe that this is one of
> reasons (perhaps the main one) that some such failures so often cause
> debuggers to be unable to find the stack pointer.
>
> I have tracked a few of those down, and have occasionally identified
> the cause (and even got it fixed!), but it is a murderous task, and
> I know of few other people who have ever succeeded.
>
> > How hard would it be to implement this for the platforms Python supports,
> > rather than for a hypothetical standards-exact platform?
>
> I have seen this effect on OSF/1, IRIX, Solaris, Linux and versions
> of Microsoft Windows.  I have never used a modern BSD, haven't used
> HP-UX since release 9, and haven't used Microsoft systems seriously
> in years (though I did hang my new laptop in its GUI fairly easily).
>
> As I say, this isn't so much a platform issue as a circumstance one.
>
>
> Regards,
> Nick Maclaren,
> University of Cambridge Computing Service,
> New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
> Email:  nmm1 at cam.ac.uk
> Tel.:  +44 1223 334761    Fax:  +44 1223 334679
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/gjcarneiro%40gmail.com
>