[Python-Dev] Subprocess destructor now logs a ResourceWarning

[Victor Stinner]

Hi,

FYI I made a tiny enhancement in the subprocess module in Python 3.6:
Popen destructor now logs a ResourceWarning if the child process is
still running when the destructor is called. Keeping a child process
in background is bad because it is likely to create zombi process,
because Python will no more read its exit status.

Martin Panter asked to add a new "detach() method" to subprocess.Popen
if you would like to "transfer the ownership" of the child process
(pid) to a different object. I'm not sure that it is needed in
practice (Popen is responsible to handle different resources, not only
the pid, maybe I didn't look closely to the issue), but it may be
required for specific code to avoid the new ResourceWarning. Please
take a look at :

   https://bugs.python.org/issue27068

Another enhancement:  ResourceWarning now also logs where "leaked"
resource was created if you enable the tracemalloc module. Example
with tracemalloc logging 10 frames (truncated output for readability):
---------------------
$ ./python -X tracemalloc=10 -m test test_sys test_os
Run tests sequentially
0:00:00 [1/2] test_sys
^C

.../Lib/subprocess.py:1023: ResourceWarning: subprocess 28856 is still running
  ResourceWarning, source=self)
Object allocated at (most recent call first):
  File "/home/haypo/prog/python/default/Lib/test/test_sys.py", lineno 702
    universal_newlines=True)
  (...)

.../Lib/test/libregrtest/main.py:343: ResourceWarning: unclosed file
<_io.TextIOWrapper name=3 encoding='UTF-8'>
  break
Object allocated at (most recent call first):
  File "/home/haypo/prog/python/default/Lib/subprocess.py", lineno 955
    self.stdout = io.TextIOWrapper(self.stdout)
  File "/home/haypo/prog/python/default/Lib/test/test_sys.py", lineno 702
    universal_newlines=True)
  File "/home/haypo/prog/python/default/Lib/test/test_sys.py", lineno 714
    out = self.c_locale_get_error_handler(encoding=':ignore')
  (...)

Test suite interrupted by signal SIGINT.
2 tests omitted:
    test_os test_sys

Total duration: 551 ms
---------------------

Please run tests using -Wd (enabled by default in debug builds) and
watch for ResourceWarning, and then report these warnings on the bug
tracker. Each ResourceWarning is likely to highlight a bug.

I just fixed the subprocess ResourceWarning in the
test.support.script_helper module (the child process is now killed on
error, and I added a context manager to cleanup all Popen resources).

Victor