[Python-ideas] shutil.runret and shutil.runout

[Mike Meyer]

On Fri, 24 Feb 2012 13:46:12 +0200
anatoly techtonik <techtonik at gmail.com> wrote:
> On Fri, Feb 24, 2012 at 2:25 PM, Mike Meyer <mwm at mired.org> wrote:
> > On Fri, 24 Feb 2012 12:10:08 +0100
> > Tarek Ziadé <ziade.tarek at gmail.com> wrote:
> >> On Fri, Feb 24, 2012 at 11:52 AM, anatoly techtonik <techtonik at gmail.com>wrote:
> >> > Hello,
> >> > subprocess is low level, cryptic, does too much, with poor usability,
> >> > i.e. "don't make me think" is not about it. I don't know about you,
> >> > but I can hardly write any subprocess call without spending at least
> >> > 5-10 meditating over the documentation. So, I propose two high level
> >> > KISS functions for shell utils (shutil) module:
> >> > runret(command)   - run command through shell, return ret code
> >> mmm you are describing subprocess.call()  here... I don't see how this new
> >> command makes thing better, besides shell=True.
> > The stated purpose of the new functions is to allow people to run
> > shell commands without thinking about them. That's a bad idea (isn't
> > most programming without thinking about it?). The first problem is
> > that it's a great way to add data injection vulnerabilities to your
> > application. It's also a good way to introduce bugs in your
> > application when asked to (for instance) process user-provided file
> > names.
> > -1
> The proposal doesn't took into account security implications, so your
> -1 is premature.

Failing to take into account security implications means the -1 isn't
premature, it's mandatory!

> I agree with your point that users should think about *security* when
> they run commands. But they should not think about how tons of
> different ways to execute their command and different combinations on
> different operating systems, *and* security implications about this.

This sounds like a documentation issue, not a code issue.

In fact, checking the shutil docs (via pydoc) turns up:

    shutil - Utility functions for copying and archiving files and directory trees.

Clearly, running commands is *not* part of this functionality, so
these new functions don't belong there.

> If users choose to trade security over simplicity, they should know
> what the risks are, and what to do if they want to avoid them. So I
> completely support the idea of shutil docs containing a user friendly
> explanation of how to exploit and how to protect (i.e. use subprocess)
> from the flaws provided by this method of execution - if they need to
> protect. Python is not a Java - it should give users a choice of
> simple API when they don't need security, and let this choice of
> shooting themselves in the foot be explicit.. and simple.

So now look at use cases. The "simple" method you propose is *only*
safe to use on a very small set of constant strings. If any of the
values in the string are supplied by the user in any way, you can't
use it. If any of the arguments contain shell meta-characters, you
either have to quote them or not use your method. Since you're
explicitly proposing passing the command to the shell, the programmer
doesn't even know which characters are meta-characters when they write
the code. This means these functions - as proposed - are more
attractive nuisances than useful utilities.

Oddly enough, I read the Julia docs on external commands between my
first answer and your reply, and their solution is both as simple as
what you want, and safe. This inspired a counter proposal:

How about adding your new function to subprocess, except instead of
passing them to the shell, they use shlex to parse them, then call
Popen with the appropriate arguments? shlex might need some work for
this.

      <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/
Independent Software developer/SCM consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org