[New-bugs-announce] [issue46464] concurrent.futures.ProcessPoolExecutor can deadlock when tcmalloc is used
Yilei Yang
report at bugs.python.org
Fri Jan 21 18:32:53 EST 2022
New submission from Yilei Yang <yileiyang9 at gmail.com>:
When Python is built and linked with tcmalloc, using ProcessPoolExecutor may deadlock. Here is a reproducible example:
$ cat t.py
from concurrent import futures
import sys
def work(iteration, item):
sys.stdout.write(f'working: iteration={iteration}, item={item}\n')
sys.stdout.flush()
for i in range(0, 10000):
with futures.ProcessPoolExecutor(max_workers=2) as executor:
executor.submit(work, i, 1)
executor.submit(work, i, 2)
$ python t.py
working: iteration=0, item=1
working: iteration=0, item=2
working: iteration=1, item=1
working: iteration=1, item=2
...
working: iteration=3631, item=1
working: iteration=3631, item=2
<hang here>
The child process fails to finish. It's more likely to reproduce when the system is busy.
With some bisect search internally, this commit https://github.com/python/cpython/commit/1ac6e379297cc1cf8acf6c1b011fccc7b3da2cbe "triggered" the deadlock threshold with tcmalloc.
----------
components: Library (Lib)
messages: 411208
nosy: yilei
priority: normal
severity: normal
status: open
title: concurrent.futures.ProcessPoolExecutor can deadlock when tcmalloc is used
versions: Python 3.10, Python 3.11, Python 3.9
_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue46464>
_______________________________________
More information about the New-bugs-announce
mailing list