[issue13238] Add shell command helpers to subprocess module

[Nick Coghlan]

Nick Coghlan <ncoghlan at gmail.com> added the comment:

Similar to what I did with walkdir, I plan to publish a shellcommand module on PyPI to give people a chance to look at the API and experiment with it without having to patch shutil in the standard library.

The only aspect I'm 100% sold on at the moment for the stdlib version is the use of a custom formatter to implement automatic invocation of shlex.quote(). It makes doing the right thing to avoid shell injection attacks (and whitespace problems in filenames) the path of least resistance: if you use shell_call/shell_check_call/shell_output and let them do the interpolation, then your call will be safe by default and the only thing you have to avoid is using an explicit conversion specifier that will bypass the quoting (i.e. '!a', '!r', '!s' or the custom '!u'). Since fear of encouraging shell injection vulnerabilities has historically been a big issue when it comes to shell invocation from the subprocess module, I think this part is mandatory. Once you do that, then '!u' is needed as a consequence, since sometimes you *will* want to pass unquoted values through in order to use standard numeric formatting options, or just to allow metacharacters to be interpreted by the shell, etc.

Interface wise, I'm still leaning towards just the _ShellFormatter class being private (due to its thread safety limitations), with ShellCommand and the three module level helpers making up the initial public API.

This isn't just speculation - by making the choice to use the *args and **kwds slots of the helper functions for string interpolation, it means they're no longer available for arguments to the Popen constructor.

In my own use case, one of the big things I want to do is invoke shell_call() with stdout and stderr redirected to a log file. Without ShellCommand exposed to let you customise the arguments to Popen, you end up having to essentially reimplement everything the helpers otherwise provide. With ShellCommand, you're able to still handle those less common use cases *without* having to reinvent the wheel on the formatting front:

    cmd = ShellCommand("cat {}", stdout=logfile, stderr=STDOUT)
    cmd.shell_call(cmd_arg)

For simple use cases where you *don't* want to tinker with the Popen args, though, the module level helpers will be the preferred option:

    data = shell_output("cat {}", cmd_arg).decode("utf-8")

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue13238>
_______________________________________