[Python-ideas] Suprocess functionality partitioning

[Chris Rebert]

On Wed, Jun 12, 2013 at 12:44 PM, Joshua Landau
<joshua.landau.ws at gmail.com> wrote:
> On 12 June 2013 20:32, anatoly techtonik <techtonik at gmail.com> wrote:
>> Something needs to be done about subprocess API, which got overly
>> complicated.
>>
>> The idea is to have:
>>   shutil.run()  - run command through shell, unsafe
>>   sys.run()    - run command directly in operating system, ?safe
>>
>> Both should be API compatible (unblocking stdin/stdout read etc.).
>> Currently, subprocess calls are unreadable in any Python code - many
>> conditions makes its behaviour hard to predict and memorize. By
>> partitioning shell and out-of-shell execution, the documentation will be
>> easier to grasp and reference to. Maybe it will be even possible to add
>> some 2D table of various subprocess states that affect behavior.
>
> Ranger (the best file manager ;P - http://ranger.nongnu.org/) has
> shown that the current method is rather useful - a simple flag to run
> in shell vs not is much easier to handle. I also, in my experience,
> appreciate the quick toggling we have for utilising the two modes. I
> also cannot agree that it would be easier to memorize the two
> functions than the one with the flag.

How do you deal with the differences? It takes a lot more than just
simply toggling the boolean to switch between the two approaches.

I fully support separating the approaches. The cross-product of
`shell` and `args` possibilities include nonsensical and deceptive
"happens to work in some cases" combinations that cause much user
confusion which could be avoided by separating them.


shell=True, isinstance(args, str) ==> works normally (can be unsafe,
but the user has explicitly requested it)

shell=False, isinstance(args, str) ==>
Works only if no arguments are being passed to the subprocess.
Users, not reading the docs and assuming that Python designed things
intuitively, very commonly get confused by this.
"/usr/bin/foo" works; "/usr/bin/foo -a bar" doesn't work, and, at
least on *nix, the error message in the latter case is approximately
"OSError: [Errno 2] No such file or directory", which doesn't make the
cause quite obvious enough and leaves the user confused, because
/usr/bin/foo really does exist.

shell=False, isinstance(args, list) ==> works normally
(so long as the user gets the tokenization right, which isn't that
hard, but plenty of people seem to mess it up, but that is all a
separate issue from the present one)

shell=True, isinstance(args, list) ==> not really useful; no known use case
Essentially an implementation artifact that's arguably a bug:
http://bugs.python.org/issue6760#msg98732
Again, confuses the user.


And Popen()'s `executable` argument just complicates this analysis even further.

Cheers,
Chris