[Python-ideas] Tulip / PEP 3156 - subprocess events

[Paul Moore]

On 20 January 2013 22:53, Andrew Bennetts <andrew at bemusement.org> wrote:
> Guido van Rossum wrote:
> […]
>> I have a more-or-less working but probably incomplete version checked
>> into the tulip repo:
>> http://code.google.com/p/tulip/source/browse/tulip/subprocess_transport.py
>>
>> Note that this completely ignores stderr -- this makes the code
>> simpler while still useful (there's plenty of useful stuff you can do
>> without reading stderr), and avoids the questions Greg Ewing brought
>> up about needing two transports (one for stdout, another for stderr).
>
> Although 3 pipes to a subprocess (stdin, stdout, stderr) is the usual
> convention, it's not the only possibility, so that configuration
> shouldn't be hard-coded.  On POSIX some programs can and do make use of
> the ability to have more pipes to a subprocess; e.g. the various *fd
> options of gnupg (--status-fd, --logger-fd, --command-fd, and so on).
> And some programs give the child process file descriptors that aren't
> pipes, like sockets (e.g. an inetd-like server that accepts a socket
> then spawns a subprocess to serve it).
>
> So I hope tulip will support these possibilities (although obviously the
> stdin/out/err style should be the convenient default).  You will be
> unsurprised to hear that Twisted does :)

My plan is to modify Guido's current code to take a subprocess.Popen
object when creating a connection to a subprocess. So you'd use the
existing API to start the process, and then tulip to interact with it.
Having said that, I have no idea if or how subprocess.Popen would
support the extra fds you are talking about. If you can show me some
sample code, I can see what would be needed to handle it. But as far
as I know, subprocess.Popen objects only have the 3 standard handles
exposed as attributes - stdin, stdout and stderr.

If you have to create your own pipes and manage them yourself in
"normal" code, then I would expect that you'd have to do the same with
tulip. That may indicate a need for (yet another) event loop API to
create a pipe which can then be used with subprocess. Or you could use
the add_reader/add_writer interfaces, at the expense of portability.

Paul

PS The above is still my plan. But at the moment, every PC in my house
seems to have decided to stop working, so I'm rebuilding PCs rather
than doing anything useful :-( Normal service will be resumed in due
course...