[Python-Dev] Usage of the multiprocessing API and object lifetime

[Victor Stinner]

Le mar. 11 déc. 2018 à 17:06, Antoine Pitrou <solipsis at pitrou.net> a écrit :
> > We are not talking about simple strings, but processes and threads.
>
> Right, but do those have an impact on the program's correctness, or
> simply on its performance (or memory consumption)?

Performance.

I made a similar change in the socketserver: server_close() now waits
until child processes (ForkingMixIn) and threads (ThreadingMixIn)
complete:

* https://bugs.python.org/issue31233
* https://bugs.python.org/issue31151

I added an opt-in option "block_on_close" to get Python 3.6 behavior
on server_close():

https://bugs.python.org/issue33540

I don't know if these changes are similar to my questions about
multiprocessing API, since socketserver didn't expose the list of
processes/threads and doesn't provide a method to wait until they
complete. Well... ForkingMixIn has an *undocumented*
collect_children() which is non-blocking by default (I added a
keyword-only 'blocking' parameter).


Another example: the close() method of an asyncio event loop doesn't
wait until threads/processes complete:

"asyncio: BaseEventLoop.close() shutdowns the executor without waiting
causing leak of dangling threads"
https://bugs.python.org/issue34037

It's unclear to me how this issue should be fixed.

Victor