[New-bugs-announce] [issue17273] multiprocessing.pool.Pool task/worker handlers are not fork safe

Arun Babu Neelicattu report at bugs.python.org
Fri Feb 22 07:20:53 CET 2013

New submission from Arun Babu Neelicattu:

The task/worker handler threads in the multiprocessing.pool.Pool class are (in accordance to posix standards) not copied over when the process containing the pool is forked.

This leads to a situation where the Pool keeps receiving tasks but the tasks never get handled. This could potentially lead to deadlocks if AsyncResult.wait() is called.

Not sure if this should be considered as a bug, or an invalid use case. However, this becomes a problem when importing modules that use pools and the main code uses multiprocessing too.

[BAD] Workaround:
Reassigning Pool._task_handler to a new instance of threading.Thread after the fork seems to work in the case highlighted in the example.

Fedora 18 
Linux 3.7.8-202.fc18.x86_64 #1 SMP Fri Feb 15 17:33:07 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

An example of this issue is shown below:

from multiprocessing import Pool, Process

def t2():
	# We expect the pool to handle this
	print('t2: Hello!')

pool = Pool()
def t1():
	# We assign a task to the pool
	print('t1: Hello!')

if __name__ == '__main__':
	# Process() forks the main process containing the pool

components: Library (Lib)
files: pool_forking.py
messages: 182647
nosy: abn
priority: normal
severity: normal
status: open
title: multiprocessing.pool.Pool task/worker handlers are not fork safe
type: behavior
versions: Python 3.3
Added file: http://bugs.python.org/file29157/pool_forking.py

Python tracker <report at bugs.python.org>

More information about the New-bugs-announce mailing list