[issue37507] multiprocessing: Add a stop() method to ForkServer

[STINNER Victor]

New submission from STINNER Victor <vstinner at redhat.com>:

While working on bpo-37421, I fixed multiprocessing tests to explicitly call _run_finaliers(), so temporary directories are removed:

New changeset 039fb49c185570ab7b02f13fbdc51c859cfd831e by Victor Stinner in branch 'master':
bpo-37421: multiprocessing tests call _run_finalizers() (GH-14527)
https://github.com/python/cpython/commit/039fb49c185570ab7b02f13fbdc51c859cfd831e

Problem: some tests started to fail when run more than once, which happens using ./python -m test -R 3:3 (hunt reference leaks). So I made a first change:

New changeset 9d40554e0da09a44a8547f3f3a2b9dedfeaf7928 by Victor Stinner in branch 'master':
bpo-37421: Fix multiprocessing get_temp_dir() finalizer (GH-14572)
https://github.com/python/cpython/commit/9d40554e0da09a44a8547f3f3a2b9dedfeaf7928

But I also had to explicitly stop the ForkServer:

New changeset 9d40554e0da09a44a8547f3f3a2b9dedfeaf7928 by Victor Stinner in branch 'master':
bpo-37421: Fix multiprocessing get_temp_dir() finalizer (GH-14572)
https://github.com/python/cpython/commit/9d40554e0da09a44a8547f3f3a2b9dedfeaf7928


I propose to add a public ForkServer.stop() method to stop the server. The method would block until the server stops. The server doesn't track its children, so if there are running child processes, they will continue to run after stop() completes. Child processes are tracked by APIs like multiprocessing Pool.

I dislike relying on implicit resource management. I prefer to handle them explicitly, to ensure that errors are properly reported if something goes wrong. Implicit resource management rely on Python destructor which may be called very late during Python finalization, while other modules are already cleaned up, and so some function may fail silently (the machineny to report issues is broken as well).


In short, I propose to rename the new _stop() method as stop() and document it properly :-)

----------
components: Library (Lib)
messages: 347351
nosy: davin, pablogsal, pitrou, vstinner
priority: normal
severity: normal
status: open
title: multiprocessing: Add a stop() method to ForkServer
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue37507>
_______________________________________