[New-bugs-announce] [issue43449] multiprocessing.Pool - crash in subprocess causes deadlock in parent

[Jamie Kirkpatrick]

New submission from Jamie Kirkpatrick <jkp at kirkconsulting.co.uk>:

When using multiprocessing.Pool.apply[_async] a crash in the subprocess that is assigned the work item results in a deadlock in the parent process.

The parent process remains blissfully unaware of the crash in the subprocess and waits for a result forever. The parent process treats this as normal since the thread running _maintain_pool handles dead processes and repopulates the pool with a replacement subprocess.

See the test-case attached. Its not clear how this case should be handled but it can be very hard to trace issues in an application where this condition arises since all information about the crashing subprocess is lost (even with debug logging for the multiprocessing module enabled).

----------
components: Library (Lib)
files: test.py
messages: 388371
nosy: jkp
priority: normal
severity: normal
status: open
title: multiprocessing.Pool - crash in subprocess causes deadlock in parent
type: behavior
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file49860/test.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43449>
_______________________________________