[Python-Dev] subprocess crossplatformness and async communication

[Daniel Stutzbach]

On Mon, Jan 26, 2009 at 4:43 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> With the vanilla subprocess Popen implmentation, the stdin.write calls
> can actually both block if the stdin buffer is full (waiting for the
> child process to clear space) and the stdout.readline call can
> definitely block (waiting for the child process to end the line).


That's true, but for the use cases presented so far, .readline() is
satisfactory, unless you have an unusual application that will fill the pipe
without sending a single newline (in which case, see below).


> So it isn't async communication in general that is the concern: it is
> *non-blocking* async communication. (Since I'm happy with the idea of
> threaded programs, I'd personally just farm the task off to some worker
> threads and make it non-blocking that way, but that approach effectively
> abandons the whole concept of non-blocking IO and it's ability to avoid
> using threads).


If you really need to communicate with multiple subprocesses (which so far
has not been suggested as a motivating example), then you can use select().

You don't need non-blocking IO to avoid using threads (although that is a
common misconception).  If a program never blocks, then it uses 100% of CPU
by definition, which is undesirable. ;-)  A program just needs select() so
it knows which file descriptors it can call os.read() or os.write() on
without blocking.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090126/d914ab3b/attachment.htm>