[Python-Dev] [Python-checkins] cpython: Switch subprocess stdin to a socketpair, attempting to fix issue #19293 (AIX

[Antoine Pitrou]

Le Tue, 22 Oct 2013 10:54:03 +0200,
Victor Stinner <victor.stinner at gmail.com> a écrit :

> Hi,
> 
> Would it be possible to use os.pipe() on all OSes except AIX?
> 
> Pipes and socket pairs may have minor differences, but some
> applications may rely on these minor differences. For example, is the
> buffer size the same? For example, in test.support, we have two
> constants: PIPE_MAX_SIZE (4 MB) and SOCK_MAX_SIZE (16 MB).

For the record, pipe I/O seems a little faster than socket I/O under
Linux:

$ ./python -m timeit -s "import os, socket; a,b = socket.socketpair(); r=a.fileno(); w=b.fileno(); x=b'x'*1000" "os.write(w, x); os.read(r, 1000)"
1000000 loops, best of 3: 1.1 usec per loop

$ ./python -m timeit -s "import os, socket; a,b = socket.socketpair(); x=b'x'*1000"
"a.sendall(x); b.recv(1000)"
1000000 loops, best of 3: 1.02 usec per loop

$ ./python -m timeit -s "import os; r, w = os.pipe(); x=b'x'*1000" "os.write(w, x); os.read(r, 1000)"
1000000 loops, best of 3: 0.82 usec per loop


Regards

Antoine.


> 
> Victor
> 
> 2013/10/22 guido.van.rossum <python-checkins at python.org>:
> > http://hg.python.org/cpython/rev/2a0bda8d283d
> > changeset:   86557:2a0bda8d283d
> > user:        Guido van Rossum <guido at dropbox.com>
> > date:        Mon Oct 21 20:37:14 2013 -0700
> > summary:
> >   Switch subprocess stdin to a socketpair, attempting to fix issue
> > #19293 (AIX hang).
> >
> > files:
> >   Lib/asyncio/unix_events.py                |  29 +++++++++-
> >   Lib/test/test_asyncio/test_unix_events.py |   7 ++
> >   2 files changed, 32 insertions(+), 4 deletions(-)
> >
> >
> > diff --git a/Lib/asyncio/unix_events.py b/Lib/asyncio/unix_events.py
> > --- a/Lib/asyncio/unix_events.py
> > +++ b/Lib/asyncio/unix_events.py
> >          if stdin == subprocess.PIPE:
> >              self._pipes[STDIN] = None
> > +            # Use a socket pair for stdin, since not all platforms
> > +            # support selecting read events on the write end of a
> > +            # socket (which we use in order to detect closing of
> > the
> > +            # other end).  Notably this is needed on AIX, and works
> > +            # just fine on other platforms.
> > +            stdin, stdin_w = self._loop._socketpair()
> >          if stdout == subprocess.PIPE:
> >              self._pipes[STDOUT] = None
> >          if stderr == subprocess.PIPE: