[Tutor] Howto display progress as cmd is being executed via subprocess ?

[eryk sun]

On Mon, Jun 13, 2016 at 4:50 PM, Ramanathan Muthaiah
<rus.cahimb at gmail.com> wrote:
>
> subprocess.check_output([cmd], stderr=subprocess.STDOUT, shell=True)
> ...
> how to combine the progressbar and subprocess code snippets to show the
> progress as the cmd is being executed.

check_output waits for the process to exit, so you can't use it with a
progress bar. However, if the program writes its progress to stdout,
then you can use `p = subprocess.Popen(args, stdout=PIPE)`, and create
a thread to read p.stdout, parse the output, and update the progress
bar.

Many programs detect when a standard stream isn't an interactive TTY
and switch to full buffering. You need to avoid this if you're
tracking progress or otherwise need to interact with a child process.
If you're on a Unix system you can use the stdbuf utility program to
try to force the child to disable buffering or use line buffering. If
that doesn't work, or you're on Windows, maybe the program has a
command-line option or environment variable setting that forces
interactive mode, such as Python's `-i` option. If all else fails, try
pexpect or winpexpect instead of subprocess. This works by making a
program think it's connected to a terminal/console.

FYI, with shell=True, you should use a command line instead of an args list.

The POSIX implementation does the following:

    if isinstance(args, (str, bytes)):
        args = [args]
    else:
        args = list(args)

    if shell:
        args = ["/bin/sh", "-c"] + args

Using just [cmd] doesn't cause a problem, but [cmd, arg1, arg2] is
probably a mistake. This passes the arguments to the shell, which sets
them as $N parameters. For example:

    >>> subprocess.call(['echo $0 $1', 'arg1', 'arg2'], shell=True)
    arg1 arg2

The Windows implementation does the following:

    if not isinstance(args, str):
        args = list2cmdline(args)

    if shell:
        startupinfo.dwFlags |= _winapi.STARTF_USESHOWWINDOW
        startupinfo.wShowWindow = _winapi.SW_HIDE
        comspec = os.environ.get("COMSPEC", "cmd.exe")
        args = '{} /c "{}"'.format (comspec, args)

Thus on Windows even [cmd] is potentially wrong with shell=True.
list2cmdline doesn't know how to escape special characters for
cmd.exe.