[New-bugs-announce] [issue44462] multiprocessing.get hangs if the pool is closed in the signal handler

[Chaitanya]

New submission from Chaitanya <chaitanya.tk17 at gmail.com>:

This is similar to https://bugs.python.org/issue22393 but the child is killed in the Pool.terminate() using SIGTERM i.e., proper way.

Run the attached script that runs an indefinite method using multiprocessing.apply_async, and in the signal handler for SIGTERM does the pool clean up (close, terminate and join), on issuing a SIGTERM (`kill <parent PID>`), the cleanup works fine, but the parent process is stuck in get() as during termination the event is not set.

Able to repro with all 3.6/3.9/3.10.


See logs below
```
[DEBUG/MainProcess] created semlock with handle 140035884298240
[DEBUG/MainProcess] created semlock with handle 140035884294144
[DEBUG/MainProcess] created semlock with handle 140035884290048
[DEBUG/MainProcess] created semlock with handle 140035884285952
[DEBUG/MainProcess] created semlock with handle 140035884281856
[DEBUG/MainProcess] created semlock with handle 140035884277760
[DEBUG/MainProcess] added worker
[INFO/ForkPoolWorker-1] child process calling self.run()
[INFO/MainProcess] <function close_thread_pools at 0x7f5ca3a20b80>
[INFO/MainProcess] <function close_thread_pools at 0x7f5ca3a20b80>
[INFO/MainProcess] <function close_thread_pools at 0x7f5ca3a20b80>
[INFO/ForkPoolWorker-1] Signals for 28462
[INFO/ForkPoolWorker-1] Handlers.SIG_DFL
[INFO/ForkPoolWorker-1] <built-in function default_int_handler>
[INFO/ForkPoolWorker-1] Handlers.SIG_DFL
[INFO/MainProcess] 28461: Caught signal 15
[INFO/MainProcess] 28461: Closing pools
[INFO/MainProcess] 28461: Terminating pools
[DEBUG/MainProcess] terminating pool
[DEBUG/MainProcess] finalizing pool
[DEBUG/MainProcess] helping task handler/workers to finish
[DEBUG/MainProcess] removing tasks from inqueue until task handler finished
[DEBUG/MainProcess] joining worker handler
[DEBUG/MainProcess] worker handler exiting
[DEBUG/MainProcess] result handler found thread._state=TERMINATE
[DEBUG/MainProcess] ensuring that outqueue is not full
[DEBUG/MainProcess] result handler exiting: len(cache)=1, thread._state=TERMINATE
[DEBUG/MainProcess] task handler got sentinel
[DEBUG/MainProcess] task handler sending sentinel to result handler
[DEBUG/MainProcess] task handler sending sentinel to workers
[DEBUG/MainProcess] task handler exiting
[DEBUG/MainProcess] terminating workers
[DEBUG/MainProcess] joining task handler
[DEBUG/MainProcess] joining result handler
[DEBUG/MainProcess] joining pool workers
[DEBUG/MainProcess] cleaning up worker 28462
[INFO/MainProcess] 28461: Joining pools
[DEBUG/MainProcess] joining pool
[INFO/MainProcess] 28461: Closed pools
```

```
(gdb) py-bt
Traceback (most recent call first):
  File "/usr/lib/python3.9/threading.py", line 312, in wait
    waiter.acquire()
  File "/usr/lib/python3.9/threading.py", line 574, in wait
    signaled = self._cond.wait(timeout)
  File "/usr/lib/python3.9/multiprocessing/pool.py", line 762, in wait
    self._event.wait(timeout)
  File "/usr/lib/python3.9/multiprocessing/pool.py", line 765, in get
    self.wait(timeout)
  File "/home/tatac/workspaces/gerrit/bwt-device-health-manager/mp_test.py", line 66, in <module>
    a.get()
(gdb) 
```

----------
components: Library (Lib)
files: mp_test.py
messages: 396151
nosy: tkc17
priority: normal
severity: normal
status: open
title: multiprocessing.get hangs if the pool is closed in the signal handler
type: behavior
versions: Python 3.10
Added file: https://bugs.python.org/file50121/mp_test.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue44462>
_______________________________________