[issue24909] Windows: subprocess.Popen: race condition for leaking inheritable handles

[STINNER Victor]

STINNER Victor added the comment:

Yeah, when stdin, stdout or stderr is a pipe, subprocess.Popen() calls CreateProcess() with the STARTF_USESTDHANDLES flag and bInheritHandles=TRUE.

This issue was fixed on UNIX for file descriptor inheritance. The fix is a major change: Python 3.4 now only creates non-inheritable file descriptors by default. See the PEP 446.

For your issue, see the "Only Inherit Some Handles on Windows" section:
https://www.python.org/dev/peps/pep-0446/#only-inherit-some-handles-on-windows

But this section is not fixed by the PEP. I was decided to fix inheritance of file descriptors and inheritance of handles separatly.

I guess that this issue is a duplicate of the issue #19575.

I'm against the idea of adding a lock inside the subprocess module. I may introduce deadlock issues. You can already put such lock in your application, to call subprocess.Popen (directly or indirectly).

> Currently, the Popen constructor will duplicate any stdout, stdin, and/or stderr handle passed in and make them inheritable, by calling DuplicateHandle. If two threads call Popen at the same time, the newly created inheritable handles will leak into the subprocess that's running being created in the opposite thread. This has consequences when two or more subprocesses are piped together and executed at the time time.

This is a race condition really specific to the subprocess module. Usually, handles are created non-inhertable on Windows, so calling CreateProcess() with bInheritHandles=TRUE is not an issue in the common case. Here the problem is that subprocess makes duplicated handles inheritable. There is a short window where a second thread can spawn a process and inherit these handles too.

The real fix is to never make duplicated handles inheritable, but instead to use the new PROC_THREAD_ATTRIBUTE_HANDLE_LIST structure. Another option to fix the issue is to use a wrapper application and send handles from the parent to the child, but this option introduces a lot of complexity since the parent has to handle two processes, not only one! Again, see the issue #19575.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24909>
_______________________________________