[Python-Dev] Questions about signal handling.

[Gregory P. Smith]

On Mon, Sep 24, 2018 at 1:20 PM Eric Snow <ericsnowcurrently at gmail.com>
wrote:

> On Mon, Sep 24, 2018 at 11:14 AM Yury Selivanov <yselivanov.ml at gmail.com>
> wrote:
> > On Fri, Sep 21, 2018 at 7:04 PM Eric Snow <ericsnowcurrently at gmail.com>
> wrote:
> > > 1. Why do we restrict calls to signal.signal() to the main thread?
> > > 2. Why must signal handlers run in the main thread?
> > > 3. Why does signal handling operate via the "pending calls" machinery
> > > and not distinctly?
> >
> > Here's my take on this:
> >
> > Handling signals in a multi-threaded program is hard. Some signals can
> > be delivered to an arbitrary thread, some to the one that caused them.
> > Posix provides lots of mechanisms to tune how signals are received (or
> > blocked) by individual threads, but (a) Python doesn't expose those
> > APIs, (b) using those APIs correctly is insanely hard.  By restricting
> > that we can only receive signals in the main thread we remove all that
> > complexity.
>
> Just to be clear, I'm *not* suggesting that we allow folks to specify
> in which Python (or kernel) thread a Python-level signal handler is
> called.
>
> The reason I've asked about signals is because of the semantics under
> subinterpreters (where there is no "main" thread).  However, I don't
> have any plans to introduce per-interpreter signal handlers.  Mostly I
> want to understand about the "main" thread restriction for the
> possible impact on subinterpreters.
>
> FWIW, I'm also mildly curious about the value of the "main" thread
> restriction currently.  From what I can tell the restriction was made
> early on and there are hints in the C code that it's no longer needed.
> I suspect we still have the restriction solely because no one has
> bothered to change it.  However, I wasn't sure so I figured I'd ask.
> :)
>

We can't change the API of the main thread being where signal handlers are
executed by default.

If a signal handler raised an exception in a daemon thread, the process
would not die when it goes uncaught (ie: why KeyboardInterrupt works).  The
daemon thread ends and the rest of the process is unaware of that.  Many
existing Python signal handlers expect to only be called from the main
thread.

If we wanted to change this, we'd probably want to have users declare which
thread(s) are allowed to execute which signal handlers at signal handler
registration time and whether they are executed by only one of those
threads or by all of those threads.  Not semantics I expect most people are
used to because I'm not aware of any other language doing that.  But I
don't see a compelling use case for implementing such complexity.  Maybe
something like that would make sense for subinterpreter delegation only?
I'm not sure.  I'd start without signals at all in subinterpreters before
making such a decision.

Python keeping signals simple has long been cited as a feature of the VM.

-gps


>
> > Restricting that signal.signal() can only be called from
> > the main thread just makes this API more consistent
>
> Yeah, that's what I thought.
>
> > (and also IIRC
> > avoids weird sigaction() behaviour when it is called from different
> > threads within one program).
>
> Is there a good place where this weirdness is documented?
>
> > Next, you can only call reentrant functions in your signal handlers.
> > For instance, printf() function isn't safe to use.  Therefore one
> > common practice is to set a flag that a signal was received and check
> > it later (exactly what we do with the pending calls machinery).
>
> We don't actually use the pending calls machinery for signals though.
> The only thing we do is *always* call PyErr_CheckSignals() before
> making any pending calls.  Wouldn't it be equivalent if we called
> PyErr_CheckSignals() at the beginning of the eval loop right before
> calling Py_MakePendingCalls()?
>
> This matters to me because I'd like to use "pending" calls for
> subinterpreters, which means dealing with signals *in*
> Py_MakePendingCalls() is problematic.  Pulling the
> PyErr_CheckSignals() call out would eliminate that problem.
>
> > Therefore, IMO, the current way we handle signals in Python is the
> > safest, most predictable, and most cross-platform option there is.
> > And changing how Python signals API works with threads in any way will
> > actually break the world.
>
> I agree that the way we deal with signals (i.e. set a flag that is
> later handled in PyErr_CheckSignals(), protected by the GIL) shouldn't
> change.  My original 3 questions do not relate to that.  Looks like
> that wasn't terribly clear. :)
>
> -eric
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180925/e52a93cd/attachment-0001.html>