[New-bugs-announce] [issue43064] Impossible to override signal handler set with add_signal_handler in forked process
report at bugs.python.org
Fri Jan 29 14:51:37 EST 2021
New submission from Selim Belhaouane <selim.belhaouane at gmail.com>:
Cannot clear signal handler set with loop.add_signal_handler in forked process with signal.signal
I'm running an async web server with uvicorn and have background processes (using multiprocessing) doing CPU-bound work. uvicorn install signal handlers with loop.add*signal_handler, if available. It also implements the common "attempt graceful shutdown on the first signal, and forcefully shutdown on the second (or more) signal" pattern. The problem I noticed was: when \_forking* at least one process, the server never gracefully shuts down, **even if I install new signal handlers in the subprocesses**.
Signal handlers installed with loop.add*signal_handler cannot be cleared in forked processes with `signal.signal` (see Experiment 1), \_unless* using `signal.SIG_DFL` or `signal.SIG_IGN` in the forked processes (see Experiment 3).
For the record:
- When installing signal handlers in the parent process with signal.signal, this is not a problem (see Experiment 2).
- When using multiprocessing with the "spawn" method, this is not a problem. Unsurprising, since signal handlers are not inherited from the parent process when using "spawn" (see Experiment 4).
- When install signal handlers in the child proceess with loop.add_signal_handler, this is not a problem (see Experiment 5).
You'll find a minimal `exp.py` file in the attached archive. There's a few tweakable parameters at the top. I just run this with `python3.X exp.py`, wait a second or two, and hit Ctrl+C.
`results.txt` shows the results of a few experiments with different set of parameters.
Note that the point at which I hit Ctrl+C is indicated by the ♥ (heart) symbol, due to some terminal weirdness, although this is quite useful in this case!
The subsections below detail what happens when Ctrl+C is hit.
You guys probably know this already, but Ctrl+C basically sends SIGINT to all processes in the process tree.
## Experiment 1
- Installs a signal handler in the parent process with loop.add_signal_handler
- Starts 3 subprocesses with "fork"
- The subprocesses install a new signal handler with signal.signal
Outcome: both the parent handler and child handlers get called (3 calls to handle_sig_worker, 4 calls to handle_sig_main)
**Expected**: I would expect a single call to handle_sig_main, and 3 calls to handle_sig_worker, as in Experiment 2 (which uses signal.signal instead of loop.add_signal_handler) or Experiment 4 (which uses "spawn" instead of "fork") or Experiment 5 ()
## Experiment 2
As Experiment 1, but signal handlers are installed in the parent process with signal.signal
Outcome: the parent handler gets called once, and the child handlers get called 3 times
## Experiment 3
Same as Experiment 1, but this time using `signal.SIG_DFL` as the callback in the child processes.
Outcome: the child processes immediately terminate, thanks to SIG_DFL, and the parent handler gets called only once!
## Experiment 4
Same as Experiment 1, but this time using "spawn" as the multiprocessing start method.
Outcome: same as Experiment 2: parent handlers gets called once, child handlers get called 3 times
## Experiment 5
Same as Experiment 1, but this time installing signal handlers in the child processes with loop.add_signal_handler.
Outcome: Same as Experiment 2.
I was able to replicate the issue with python 3.7, 3.8, 3.9 and 3.10 (didn't even try 3.6) on Ubuntu 20.04. I obtained python via `apt`.
nosy: asvetlov, selimb, yselivanov
title: Impossible to override signal handler set with add_signal_handler in forked process
versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file49774/python-async-signal-bug.zip
Python tracker <report at bugs.python.org>
More information about the New-bugs-announce