[New-bugs-announce] [issue31863] Inconsistent returncode/exitcode for terminated child processes on Windows

[Akos Kiss]

New submission from Akos Kiss <akosthekiss at gmail.com>:

I've been working with various approaches for running and terminating subprocesses on Windows and I've obtained surprisingly different results if I used different modules and ways of termination. Here is the script I wrote, it uses the `subprocess` and the `multiprocessing` modules for starting new subprocesses, and process termination is performed either by the modules' own `terminate` functions or by `os.kill`.

```py
import multiprocessing
import os
import signal
import subprocess
import sys
import time

def kill_with_os_kill(proc):
    print('kill with os.kill(pid,SIGTERM)')
    os.kill(proc.pid, signal.SIGTERM)

def kill_with_terminate(proc):
    print('kill child with proc.terminate()')
    proc.terminate()

def run_and_kill_subprocess(killfn, procarg):
    print('run subprocess child with %s' % procarg)
    with subprocess.Popen([sys.executable, __file__, procarg]) as proc:
        time.sleep(1)
        killfn(proc)
        proc.wait()
    print('child terminated with %s' % proc.returncode)

def run_and_kill_multiprocessing(killfn, procarg):
    print('run multiprocessing child with %s' % procarg)
    proc = multiprocessing.Process(target=childmain, args=(procarg,))
    proc.start()
    time.sleep(1)
    killfn(proc)
    proc.join()
    print('child terminated with %s' % proc.exitcode)

def childmain(arg):
    print('child process started with %s' % arg)
    while True:
        pass

if __name__ == '__main__':
    if len(sys.argv) < 2:
        print('parent process started')
        run_and_kill_subprocess(kill_with_os_kill, 'subprocess-oskill')
        run_and_kill_subprocess(kill_with_terminate, 'subprocess-terminate')
        run_and_kill_multiprocessing(kill_with_os_kill, 'multiprocessing-oskill')
        run_and_kill_multiprocessing(kill_with_terminate, 'multiprocessing-terminate')
    else:
        childmain(sys.argv[1])
```

On macOS, everything works as expected (and I think that Linux will behave alike):

```
$ python3 killtest.py 
parent process started
run subprocess child with subprocess-oskill
child process started with subprocess-oskill
kill with os.kill(pid,SIGTERM)
child terminated with -15
run subprocess child with subprocess-terminate
child process started with subprocess-terminate
kill child with proc.terminate()
child terminated with -15
run multiprocessing child with multiprocessing-oskill
child process started with multiprocessing-oskill
kill with os.kill(pid,SIGTERM)
child terminated with -15
run multiprocessing child with multiprocessing-terminate
child process started with multiprocessing-terminate
kill child with proc.terminate()
child terminated with -15
```

But on Windows, I got:

```
>py -3 killtest.py
parent process started
run subprocess child with subprocess-oskill
child process started with subprocess-oskill
kill with os.kill(pid,SIGTERM)
child terminated with 15
run subprocess child with subprocess-terminate
child process started with subprocess-terminate
kill child with proc.terminate()
child terminated with 1
run multiprocessing child with multiprocessing-oskill
child process started with multiprocessing-oskill
kill with os.kill(pid,SIGTERM)
child terminated with 15
run multiprocessing child with multiprocessing-terminate
child process started with multiprocessing-terminate
kill child with proc.terminate()
child terminated with -15
```

Notes:
- On Windows with `os.kill(pid, sig)`, "sig will cause the process to be unconditionally killed by the TerminateProcess API, and the exit code will be set to sig." I.e., it is not possible to detect on Windows whether a process was terminated by a signal or it exited properly, because `kill` does not actually raise a signal and no Windows API allows to differentiate between proper or forced termination.
- The `multiprocessing` module has a workaround for this by terminating the process with a designated exit code (`TERMINATE = 0x10000`) and checking for that value afterwards, rewriting it to `-SIGTERM` if found. The related documentation is a bit misleading, as `exitcode` is meant to have "negative value -N [which] indicates that the child was terminated by signal N" -- however, if the process was indeed killed with `SIGTERM` (and not via `terminate`), then `exitcode` will be `SIGTERM` and not `-SIGTERM` (see above). (The documentation of `terminate` does not clarify the situation much by stating that "on Windows TerminateProcess() is used", since it does not mention the special exit code -- and well, it's not even a signal after all, so it's not obvious whether negative or positive exit code is to be expected.)
- The `subprocess` module choses the quite arbitrary exit code of 1 and documents that "negative value -N indicates that the child was terminated by signal N" is POSIX only, not mentioning anything about what to expect on Windows.

Long story short: on Windows, the observable exit code of a forcibly terminated child process is quite inconsistent even across standard modules and termination methods, unlike on other (Linux/macOS) platforms. I think that having results consistent with `os.kill(,SIGTERM)` would be desirable even if that means non-negative values.

I'm willing to post a PR if the issue is deemed to be valid.

----------
components: Library (Lib), Windows
messages: 304920
nosy: Akos Kiss, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Inconsistent returncode/exitcode for terminated child processes on Windows
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue31863>
_______________________________________