[docs] [issue27050] Demote run() below the high level APIs in subprocess docs

Akira Li report at bugs.python.org
Wed Jun 8 11:57:08 EDT 2016


Akira Li added the comment:

> setting "universal_newlines=True" switches to UTF-8 encoded text pipes

It uses locale.getpreferredencoding(False) encoding -- something like cp1252,cp1251,etc on Windows, and UTF-8 on *nix with proper locale settings.

It is ASCII (C/POSIX locale default) if the locale is not set in cron, ssh, init.d scripts, etc.

If you need a different character encoding, you could use (instead of universal_newlines=True):

  pipe = io.TextIOWrapper(process.stdout, encoding=character_encoding)

A better spelling for universal_newlines=True would be text_mode=True.

A summary table (like in itertools' module) would be nice.

check_output() name is unfortunate but it does the right thing and it is not hard to use for a beginner --
once somebody discovers it e.g., via "Running shell command from Python and capturing the output" Stack Overflow question
http://stackoverflow.com/questions/4760215/running-shell-command-from-python-and-capturing-the-output

Compare:

  output = check_output([sys.executable, '-c', 'print("abc")'])

and

  output = run([sys.executable, '-c', 'print("abc)'], stdout=PIPE).stdout

The latter command doesn't raise exception if the child process fails.
A beginner has to know about check=True to do the right thing:

  output = run([sys.executable, '-c', 'print("abc")'], stdout=PIPE, check=True).stdout

It is easier to refer to check_output() if someone asks "How do I get command's output in Python?"


I wish call() did what check_call() does and the current call() behavior would be achieved by
the opposite parameter e.g. no_raise_on_status=False being the default:

   rc = call(command, no_raise_on_status=True)

If we can't change the interface then check_call() is the answer to "Calling an external command in Python" question
http://stackoverflow.com/questions/89228/calling-an-external-command-in-python

  - check_call(command) -- run command, raise if it fails
  - output = check_output(command) -- get command's output, raise if it fails.
      To pass *data* to the command via its standard input, pass input=data.
      To get/pass text (Unicode) instead of bytes, pass universal_newlines=True
  - check_call("a -- *.jpg | b 2>&1 >output | c", shell=True) -- run a shell command as is
      It is a pity that a list argument such as ["ls", "-l"] is allowed with shell=True

These cover the most common operations with a subprocess.

Henceforth, run() is more convenient if we don't need to interact with the child process while it is running.

For example, if  we introduce the word PIPE (a magic object in the kernel that connects processes) then
to capture both standard and error streams of the command:

  cp = run(command, stdout=PIPE, stderr=PIPE)
  output, errors = cp.stdout, cp.stderr

run() allows to get the output and to get the exit status easily: cp.returncode.
Explicit cp.stdout_text, cp.stdout_bytes regardless the text mode would be nice.

To interact with a child process while it is running, Popen() have to be used directly.
There could be buffering and other issues (tty vs. pipe), see "Q: Why not just use a pipe (popen())?"
http://pexpect.readthedocs.io/en/latest/FAQ.html#whynotpipe

Working with both stdout/stderr or a non-blocking read require threads or asyncio, fcntl, etc.

A couple of words should be said about killing a command started with shell=True.
(to kill a process tree: set start_new_session=True parameter and call os.killpg()).
timeout option doesn't work in this case (it uses Popen.kill()).
check_output() unlike check_call() may wait for grandchildren if they inherit the pipe.
Mention Job object on Windows e.g.,
http://stackoverflow.com/questions/23434842/python-how-to-kill-child-processes-when-parent-dies

----------
nosy: +akira

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27050>
_______________________________________


More information about the docs mailing list