[issue9205] Parent process hanging in multiprocessing if children terminate unexpectedly

[Greg Brockman]

New submission from Greg Brockman <gdb at ksplice.com>:

I have recently begun using multiprocessing for a variety of batch
jobs.  It's a great library, and it's been quite useful.  However, I have been bitten several times by situations where a worker process in a Pool will unexpectedly die, leaving multiprocessing hanging in a wait.  A simple example of this is produced by the following:
"""
#!/usr/bin/env python
import multiprocessing, sys
def foo(x):
  sys.exit(1)
multiprocessing.Pool(1).apply(foo, [1])
"""
The child will exit and the parent will hang forever.  A similar occurrence happens if one pushes C-c while a child process is running (this special case is noted in http://bugs.python.org/issue8296) or killed by a signal.

Attached is a patch to handle unexpected terminations of children
processes and prevent the parent process from hanging.  A test case is included.  (Developed and tested on 64-bit Ubuntu.)  Please let me know what you think.  Thanks!

----------
components: Library (Lib)
files: termination.patch
keywords: patch
messages: 109585
nosy: gdb
priority: normal
severity: normal
status: open
title: Parent process hanging in multiprocessing if children terminate unexpectedly
type: behavior
versions: Python 2.6, Python 2.7
Added file: http://bugs.python.org/file17905/termination.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9205>
_______________________________________