[issue10482] subprocess and deadlock avoidance

[Glenn Linderman]

Glenn Linderman <v+python at g.nevcal.com> added the comment:

Looking at the code the way I've used it in my modified server.py:

            stderr = []
            stderr_thread = threading.Thread(target=self._readerthread,
                                             args=(p.stderr, stderr))
            stderr_thread.daemon = True
            stderr_thread.start()

            self.log_message("writer: %s" % str( nbytes ))
            stdin_thread = threading.Thread(target=self._writerthread,
                                            args=(self.rfile, p.stdin, nbytes))
            stdin_thread.daemon = True
            stdin_thread.start()

and later

            stderr_thread.join()
            stdin_thread.join()

            p.stderr.close()
            p.stdout.close()

            if stderr:
                stderr = stderr[ 0 ].decode("UTF-8")

It seems like this sort of code (possibly passing in the encoding) could be bundled back inside subprocess (I borrowed it from there).

It also seems from recent discussion on npopdev that the cheat-sheet "how to replace" other sys and os popen functions would be better done as wrapper functions for the various cases.  Someone pointed out that the hard cases probably aren't cross-platform, but that currently the easy cases all get harder when using subprocess than when using the deprecated facilities.  They shouldn't.  The names may need to be a bit more verbose to separate the various use cases, but each use case should remain at least as simple as the prior function.

So perhaps instead of just  subprocess.PIPE  to select particular handling for stdin, stdout, and stderr, subprocess should implement some varieties to handle attaching  different types of reader and writer threads to the handles... of course, parameters need to come along for the ride too: maybe the the additional variations would be object references with parameters supplied, instead of just a manifest constant like .PIPE.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10482>
_______________________________________