[Python-Dev] Questions about signal handling.

[Victor Stinner]

Please don't rely on this ugly API. *By design*, Py_AddPendingCall()
tries 100 times to acquire the lock: if it fails to acquire the lock,
it does notthing... your callback is ignored...

By the way, recently, we had to fix yet another bug in signal
handling. A new function has been added:

void
_PyEval_SignalReceived(void)
{
    /* bpo-30703: Function called when the C signal handler of Python gets a
       signal. We cannot queue a callback using Py_AddPendingCall() since
       that function is not async-signal-safe. */
    SIGNAL_PENDING_CALLS();
}

If you want to exchange commands between two interpreters, which I see
as two threads, I suggest to use two queues and something to consume
queues.

Victor
Le lun. 24 sept. 2018 à 22:23, Eric Snow <ericsnowcurrently at gmail.com> a écrit :
>
> On Mon, Sep 24, 2018 at 11:14 AM Yury Selivanov <yselivanov.ml at gmail.com> wrote:
> > On Fri, Sep 21, 2018 at 7:04 PM Eric Snow <ericsnowcurrently at gmail.com> wrote:
> > > 1. Why do we restrict calls to signal.signal() to the main thread?
> > > 2. Why must signal handlers run in the main thread?
> > > 3. Why does signal handling operate via the "pending calls" machinery
> > > and not distinctly?
> >
> > Here's my take on this:
> >
> > Handling signals in a multi-threaded program is hard. Some signals can
> > be delivered to an arbitrary thread, some to the one that caused them.
> > Posix provides lots of mechanisms to tune how signals are received (or
> > blocked) by individual threads, but (a) Python doesn't expose those
> > APIs, (b) using those APIs correctly is insanely hard.  By restricting
> > that we can only receive signals in the main thread we remove all that
> > complexity.
>
> Just to be clear, I'm *not* suggesting that we allow folks to specify
> in which Python (or kernel) thread a Python-level signal handler is
> called.
>
> The reason I've asked about signals is because of the semantics under
> subinterpreters (where there is no "main" thread).  However, I don't
> have any plans to introduce per-interpreter signal handlers.  Mostly I
> want to understand about the "main" thread restriction for the
> possible impact on subinterpreters.
>
> FWIW, I'm also mildly curious about the value of the "main" thread
> restriction currently.  From what I can tell the restriction was made
> early on and there are hints in the C code that it's no longer needed.
> I suspect we still have the restriction solely because no one has
> bothered to change it.  However, I wasn't sure so I figured I'd ask.
> :)
>
> > Restricting that signal.signal() can only be called from
> > the main thread just makes this API more consistent
>
> Yeah, that's what I thought.
>
> > (and also IIRC
> > avoids weird sigaction() behaviour when it is called from different
> > threads within one program).
>
> Is there a good place where this weirdness is documented?
>
> > Next, you can only call reentrant functions in your signal handlers.
> > For instance, printf() function isn't safe to use.  Therefore one
> > common practice is to set a flag that a signal was received and check
> > it later (exactly what we do with the pending calls machinery).
>
> We don't actually use the pending calls machinery for signals though.
> The only thing we do is *always* call PyErr_CheckSignals() before
> making any pending calls.  Wouldn't it be equivalent if we called
> PyErr_CheckSignals() at the beginning of the eval loop right before
> calling Py_MakePendingCalls()?
>
> This matters to me because I'd like to use "pending" calls for
> subinterpreters, which means dealing with signals *in*
> Py_MakePendingCalls() is problematic.  Pulling the
> PyErr_CheckSignals() call out would eliminate that problem.
>
> > Therefore, IMO, the current way we handle signals in Python is the
> > safest, most predictable, and most cross-platform option there is.
> > And changing how Python signals API works with threads in any way will
> > actually break the world.
>
> I agree that the way we deal with signals (i.e. set a flag that is
> later handled in PyErr_CheckSignals(), protected by the GIL) shouldn't
> change.  My original 3 questions do not relate to that.  Looks like
> that wasn't terribly clear. :)
>
> -eric
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com