[Tutor] subprocess module: when to _NOT_ use shell=True

[eryksun]

On Sun, Mar 10, 2013 at 12:56 PM,  <akleider at sonic.net> wrote:
> I've not found anywhere a clear explanation of when not to
> set shell=True. If the command line must be interpreted by
> the shell then clearly this must be set. So the question
> that comes up is why not set it always?

Using the shell can be a security risk for untrusted commands, as
described in the 3.3 docs for shlex.quote:

http://docs.python.org/3/library/shlex#shlex.quote

> I came up with the following script that indicates that
> the shell looks at only the first string in the array if
> the first parameter is an array rather than a string.
> Switching between cmd being a string vs an array and shell
> being set or not set gives 4 possibilities.
>
> cmd = ["ls", "-l"]
> # cmd = "ls -l"

On a POSIX system, when you use an ags string instead of a list with
Popen, it just adds the string to a list, which varies depending on
the "shell" argument.

Starting a new process using fork/exec hasn't fundamentally changed
since the early Unix systems in the 1970s. A child process is forked
and executes a new process image, with the given arguments in an array
of pointers to strings. Popen uses uses os.fork and os.execvp (or
os.execvpe if you supply an environment).

http://docs.python.org/2/library/os#os.fork
http://docs.python.org/2/library/os#os.execvp

http://en.wikipedia.org/wiki/Exec_%28operating_system%29

If shell=False, use the args list ["ls", "-l"]. Otherwise, if you use
an args string, Popen creates the list ["ls -l"], and execvp will look
for a file named "ls -l". Here's a silly example:

    >>> import os
    >>> from subprocess import Popen

    >>> os.environ['PATH'] += ':.'
    >>> open('ls -l', 'w').write('''\
    ... #!/bin/bash
    ... echo silliness''')
    >>> os.chmod('ls -l', 0700)
    >>> p = Popen('ls -l')
    >>> silliness

If shell=True and the command is the string "ls -l", Popen uses the
args list ["/bin/sh", "-c", "ls -l"]. This is equivalent to running
the following:

    /bin/sh -c 'ls -l'

This will work as expected. If you instead use the list ["ls", "-l"],
Popen uses the args list ["/bin/sh", "-c", "ls", "-l"], which is
equivalent to running the following:

    /bin/sh -c ls -l

You can verify that the above doesn't work (the '-l' option isn't
passed to ls). Here's an example to echo the parameters:

    >>> open('tmp.sh', 'w').write('''
    ... #!/bin/bash
    ... echo $0, $1, $2''')
    >>> os.chmod('tmp.sh', 0700)
    >>> env = {'PATH':'.'}

    >>> p = Popen('tmp.sh p1 p2', shell=True, env=env)
    >>> ./tmp.sh, p1, p2

That worked fine, but this fails:

    >>> p = Popen(['tmp.sh','p1','p2'], shell=True, env=env)
    >>> ./tmp.sh, ,