[Python-Dev] A question about the subprocess implementation

[Mike Meyer]

On Sat, 7 Jan 2012 21:25:37 +0000 (UTC)
Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:

> The subprocess.Popen constructor takes stdin, stdout and stderr keyword
> arguments which are supposed to represent the file handles of the child process.
> The object also has stdin, stdout and stderr attributes, which one would naively
> expect to correspond to the passed in values, except where you pass in e.g.
> subprocess.PIPE (in which case the corresponding attribute would be set to an
> actual stream or descriptor).
> 
> However, in common cases, even when keyword arguments are passed in, the
> corresponding attributes are set to None. The following script

Note that this is documented behavior for these attributes.

> This seems to me to contradict the principle of least surprise. One
> would expect, when an file-like object is passed in as a keyword
> argument, that it be placed in the corresponding attribute.

Since the only reason they exist is so you can access your end of a
pipe, setting them to anything would seem to be a bug. I'd argue that
their existence is more a pola violation than them having the value
None. But None is easier than a call to hasattr.

> That way, if one wants to do p.stdout.close() (which is necessary in
> some cases), one doesn't hit an AttributeError because NoneType has
> no attribute 'close'.

You can close the object you passed in if it wasn't PIPE. If you
passed in PIPE, the object has to be exposed some way, otherwise you
*can't* close it.

This did raise one interesting question, which will go to ideas...

       <mike


-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/
Independent Software developer/SCM consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org



> import os
> from subprocess import Popen, PIPE
> import tempfile
> 
> cmd = 'ls /tmp'.split()
> 
> p = Popen(cmd, stdout=open(os.devnull, 'w+b'))
> print('process output streams: %s, %s' % (p.stdout, p.stderr))
> p = Popen(cmd, stdout=tempfile.TemporaryFile())
> print('process output streams: %s, %s' % (p.stdout, p.stderr))
> 
> prints
> 
> process output streams: None, None
> process output streams: None, None
> 
> under both Python 2.7 and 3.2. However, if subprocess.PIPE is passed in, then
> the corresponding attribute *is* set: if the last four lines are changed to
> 
> p = Popen(cmd, stdout=PIPE)
> print('process output streams: %s, %s' % (p.stdout, p.stderr))
> p = Popen(cmd, stdout=open(os.devnull, 'w+b'), stderr=PIPE)
> print('process output streams: %s, %s' % (p.stdout, p.stderr))
> 
> then you get
> 
> process output streams: <open file '<fdopen>', mode 'rb' at 0x2088660>, None
> process output streams: None, <open file '<fdopen>', mode 'rb' at 0x2088e40>
> 
> under Python 2.7, and
> 
> process output streams: <_io.FileIO name=3 mode='rb'>, None
> process output streams: None, <_io.FileIO name=5 mode='rb'>
> 
> This seems to me to contradict the principle of least surprise. One would
> expect, when an file-like object is passed in as a keyword argument, that it be
> placed in the corresponding attribute. That way, if one wants to do
> p.stdout.close() (which is necessary in some cases), one doesn't hit an
> AttributeError because NoneType has no attribute 'close'.
>