[New-bugs-announce] [issue37740] Python 3.7 sh hangs when using in threads, forking and logging

[mzbuild]

New submission from mzbuild <svairavan at mz.com>:

Hi,
We use the sh module in a threaded context to execute shell commands. When we were migrating to python 3 from python 2 we encountered some commands hanging. We created a test script that recreates the issue we are seeing.
`import sh
import logging
import concurrent.futures.thread

def execmd(exe):
print(sh.ls())
execution_pool = concurrent.futures.thread.ThreadPoolExecutor(20)
i = 0
thread_results = []
while i<500:
i+= 1
thread_results.append(execution_pool.map(execmd, ['ls']))

execution_pool.shutdown()

When running this in python 3.7 it hangs but in python 3.6 it works fine. We think it is releated to this issue https://bugs.python.org/issue36533. Installing the latest 3.7.4 didn't fix the issue.  The sh module uses logging and forking and the top script uses threading so we think there is a locking issue with 3.7.  Any help would be great.

----------
components: Library (Lib)
messages: 348869
nosy: cagney, ebizzinfotech, gregory.p.smith, hugh, lukasz.langa, mzbuild, ned.deily
priority: normal
severity: normal
status: open
title: Python 3.7 sh hangs when using in threads, forking and logging
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue37740>
_______________________________________