[Python-Dev] Status of PEP 3145 - Asynchronous I/O for subprocess.popen

[Josiah Carlson]

On Fri, Mar 28, 2014 at 12:42 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> On 3/28/2014 12:45 PM, Josiah Carlson wrote:
>
>> If it makes you feel any better, I spent an hour this morning building a
>> 2-function API for Linux and Windows, both tested, not using ctypes, and
>> not even using any part of asyncio (the Windows bits are in msvcrt and
>> _winapi). It works in Python 3.3+. You can see it here:
>> http://pastebin.com/0LpyQtU5
>>
>
> Thank you. The docs gave me the impression that I could simply write
> proc.stdin and read proc.stdout. I failed with even a simple echo server
> (on Windows) and your code suggests why. So it does not get lost, I
> attached your code to
>
> http://bugs.python.org/issue18823
>
> My interest is with Idle. It originally ran user code in the same process
> as the Shell and Editor code. Then Guido added an option to os.spawn a
> separate process and communicate through a socket connection and the option
> became the default with same process (requested by -N on the command line)
> as a backup option. 3.2 switched to using subprocess, but still with a
> socket. The problem is that the socket connection intermittently fails.
> Firewalls are, or at least used to be one possible cause, but there are
> others -- unknown. (While it works, the suggestion to restart with -N is a
> mystery to people who have never seen a command line.) This is one of the
> biggest sources of complaints about Idle. A pipe connection method that
> always worked on Windows, *x, and Mac would be great in itself and would
> also allow code simplification by removing the -n option. (Roger Serwy has
> suggested the latter as having two modes makes patching trickier.)
>
> The current socket connection must be non-blocking. Even though the exec
> loop part of the Shell window waits for a response after sending a user
> statement, everything else is responsive. One can select text in the
> window, use the menus, or switch to another window. So Idle definitely
> needs non-blocking write and read.
>
> In my ignorance, I have no idea whether the approach in your code or that
> in Viktor's code is better. Either way, I will appreciate any help you
> give, whether by writing, reviewing, or testing, to make communication with
> subprocesses easier and more dependable.


One of my other use-cases for this was using this in *my* editor (PyPE),
which I wrote (in 2003) because I lost work in Idle. This lost work was due
to the same-process interpreter crashing during an interactive session.
IIRC, this is partly what pushed Guido to have Idle use os.spawn() +
socket. I ended up using wxPython's built-in external process support at
the time, but that's obviously not useful in core Python with Idle :P

This is all coming back full circle. :)

 - Josiah
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140328/406cc8ba/attachment.html>