[Python-ideas] Tulip / PEP 3156 - subprocess events

[Guido van Rossum]

On Wed, Jan 16, 2013 at 7:12 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> I've so far been lurking on the tulip/async discussions, as although
> I'm interested, I have no specific need for writing high-performance
> network code.
>
> However, I hit a use case today which seems to me to be ideal for an
> async-style approach, and yet I don't think it's covered by the
> current PEP. Specifically, I am looking at monitoring a
> subprocess.Popen object. This is basically an IO loop, but monitoring
> the 3 pipes to the subprocess (well, only stdout and stderr in my
> case...). Something like add_reader/add_writer would be fine, except
> for the fact that (a) they are documented as low-level not for the
> user, and (b) they don't work in all cases (e.g. in a select-based
> loop on Windows).
>
> I'd like PEP 3156 to include some support for waiting on IO from (one
> or more) subprocesses like this in a cross-platform way. If there's
> something in there to do this at the moment, that's great, but it
> wasn't obvious to me when I looked...

This is a great use case. The right approach would probably be to
define a new Transport (and an event loop method to create one) that
wraps pipes going into and out of a subprocess. The new method would
have a standard API (probably similar to that of subprocess), whereas
there would be different implementations of the Transport based on
platform and event loop implementation (similar to the way the
subprocess module has quite different implementations).

Can you check out the Tulip source code (code.google.com/p/tulip) and
come up with a patch to do this? I'll gladly review it. It's fine to
only cover the UNIX case for now.

-- 
--Guido van Rossum (python.org/~guido)