[Python-Dev] subprocess crossplatformness and async communication

[Nick Craig-Wood]

On Sat, Jan 24, 2009 at 07:58:40AM -0800, Guido van Rossum wrote:
> I'm confused. The subprocess already allows reading/writing its
> stdin/stdout/stderr, and AFAIK it's a platform-neutral API. I'm sure
> there's something missing, but your post doesn't make it clear what
> exactly, and the recipe you reference is too large to digest easily.
> Can you explain what it is that the current subprocess does't have
> beyond saying "async communication" (which could mean many things to
> many people)?

The main problem with subprocess is that it doesn't work if you want
to have a conversation / interact with your child process.

subprocess works very well indeed for this case :-

  run child
  send stuff to stdin
  child reads stdin and writes stdout
  child exits
  read stuff from stdout

But for the conversational case (eg using it to do a remote login) it
doesn't work at all :-

  run child
  send stuff to stdin
  child reads stdin and writes stdout
  read stuff from stdout
  send stuff to stdin
  child reads stdin and writes stdout
  read stuff from stdout
  send stuff to stdin
  child reads stdin and writes stdout
  read stuff from stdout
  child exits

In subprocess "read stuff from stdout" means read stdout until the
other end closes it, not read what is available and return it, so it
blocks on reading the first reply and never returns.

Hence Anatoly's request for "async communication" and the existence of
that recipe.

  http://code.activestate.com/recipes/440554/

I've spend quite a lot of time explaning this to people on
comp.lang.python.  I usually suggest either the recipe as suggested by
Anatoly or if on unix the pexpect module.  There are other solutions I
know of, eg in twisted and wxPython.

I heard rumours of a pexpect port to Windows but I don't know how far
that has progressed.

A cross platform async subprocess would indeed be a boon!

-- 
Nick Craig-Wood <nick at craig-wood.com> -- http://www.craig-wood.com/nick