[Python-Dev] socketserver ForkingMixin waiting for child processes

[Victor Stinner]

Hi,

The first bug was that test_socketserver "leaked" child processes: it
means that socketserver API creates zombie processes depending how
long the child processes take to complete.

If you want to backport my change waiting until child processes
complete, you need to fix the bug differently, so test_socketserver
doesn't leak processes (ex: wait until child processes complete in
test_socketserver).


2017-08-12 0:34 GMT+02:00 Ryan Smith-Roberts <rmsr at lab.net>:
> I agree that blocking shutdown by default isn't a good idea. A child will
> eventually get indefinitely stuck on a nonresponsive connection and hang the
> whole server. This behavior change is surprising and should be reverted in
> master, and definitely not backported.

Right, a child process can hang. The question is how to handle such
case. I see 3 choices:

* Do nothing: Python 3.6 behaviour
* Blocking wait until the child process completes
* Wait a few seconds and then kill the process after a dead line

In Python 3.6, the process is stuck and continue to run even after
shutdown() and server_close(). That's surprising and doesn't seem
right to me.


> As for block-timeout or block-timeout-kill, waiting more than zero seconds
> in server_close() should be optional, because you're right that the best
> timeout is circumstantial.

Maybe we need to add a new method to wait N seconds until children
completes, then send SIGTERM, and finally send SIGKILL if children
take longer than N seconds to complete.

So the developer becomes responsible of killing child processes.

In the Apache world, it's called the "graceful" stop:
https://httpd.apache.org/docs/2.4/stopping.html


> Since ThreadingMixIn also leaks threads,
> server_close() could grow a timeout flag (following the socket module
> timeout convention) and maybe a terminate boolean.

Oh, I didn't know that ThreadingMixIn also leaks threads.

That's a similar but different issue. It's not easily possible to
"kill" a thread.


> Plus it can't be backported to the feature-freeze branches.

IMHO leaking zombie processes is a bug, but I'm ok to keep bugs and
only fix tests.

Victor