[Python-ideas] Tulip / PEP 3156 - subprocess events

[Guido van Rossum]

On Fri, Jan 18, 2013 at 2:48 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 18 January 2013 22:22, Guido van Rossum <guido at python.org> wrote:
>>> The protocol part is then about adapting the transport API to
>>> coroutine friendly readlines/writelines API (the part that Guido
>>> points out needs more detail in
>>> http://www.python.org/dev/peps/pep-3156/#coroutines-and-protocols)
>>>
>>> As a rough untested sketch (the buffering here could likely be a lot smarter):
>>
>> I have a more-or-less working but probably incomplete version checked
>> into the tulip repo:
>> http://code.google.com/p/tulip/source/browse/tulip/subprocess_transport.py
>
> Ha! You beat me to it.
>
> OK, looking at your code, I see that you freely used the
> add_reader/add_writer functions and friends, and the fact that the
> Unix selectors handle pipes as well as sockets. With the freedom to do
> that, your code looks both reasonable and pretty straightforward. I
> was having trouble getting past the fact that this approach wouldn't
> work on Windows, and confusing "nonportable" with "not allowed". My
> apologies. You kept telling me that writing the code for Unix would be
> helpful, but I kept thinking in terms of writing code that worked on
> Unix but with portability to Windows in mind, which completely misses
> the point. I knew that the transport/protocol code I'd end up writing
> would look something like this, but TBH I'd not seen that as the
> interesting part of the problem...

Glad you've got it now!

> BTW, to avoid duplication of the fork/exec stuff, I would probably
> have written the transport to take a subprocess.Popen object as its
> only argument, then hooked up self._wstdin to popen.stdin and
> self._rstdout to popen.stdout. That requires the user to have created
> the Popen object with those file descriptors as pipes (I don't know if
> it's possible to introspect a Popen object to check that) but avoids
> duplicating the subprocess logic. I can probably fairly quickly modify
> your code to demonstrate, but it's late and I don't want to start
> booting my Unix environment now, so it'll have to wait till tomorrow
> :-)

I would love for you to create that version. I only checked it in so I
could point to it -- I am not happy with either the implementation,
the API spec, or the unit test...

-- 
--Guido van Rossum (python.org/~guido)