Re: [Python-ideas] [Python-Dev] subprocess not escaping "^" on Windows
Redirecting this part of the conversation to python-ideas. On Mon, Jan 8, 2018 at 3:17 AM, Christian Tismer <tismer@stackless.com> wrote:
As a side note: In most cases where shell=True is found, people seem to need evaluation of the PATH variable. To my understanding,
from subprocess import call call(("ls",))
works in Linux, but (with dir) not in Windows. But that is misleading because "dir" is a builtin command but "ls" is not. The same holds for "del" (Windows) and "rm" (Linux).
That's exactly what shell=True is for - if you want a shell feature, you use the shell. What exactly would emulate_shell do? Would it simply do a $PATH or %PATH% search, but otherwise function as shell=False? Would it process redirection? Would it handle interpolations? I think not, from your description:
Perhaps it would be a good thing to emulate the builtin programs in python by some shell=True replacement (emulate_shell=True?) to match the normal user expectations without using the shell?
but it becomes difficult to draw the line. For instance, with emulate_shell=True, what would you do with all the sh/bash built-ins: https://www.gnu.org/software/bash/manual/html_node/Bourne-Shell-Builtins.htm... https://www.gnu.org/software/bash/manual/html_node/Bash-Builtins.html I'm thinking especially of the commands where bash has its own handling of something that could otherwise be found in $PATH, like pwd, time, and echo, but shells can do a lot of other things too. When do you actually want to execute a shell built-in from Python but without using the shell itself? You give the example of ls/dir, but if that ever comes up in real-world code, I'd toss it out and recommend a cross-platform os.listdir or equivalent. There are plenty of times I've wanted a really quick way to redirect a standard stream from Python, but that isn't part of what you're recommending. Can you give a real-world example that would be improved by this? I know this was just a side note in your original, but I'd like to hear more about what would make it useful. ChrisA
Hi Chris, On 07.01.18 18:18, Chris Angelico wrote:
Redirecting this part of the conversation to python-ideas.
On Mon, Jan 8, 2018 at 3:17 AM, Christian Tismer <tismer@stackless.com> wrote:
As a side note: In most cases where shell=True is found, people seem to need evaluation of the PATH variable. To my understanding,
from subprocess import call call(("ls",))
works in Linux, but (with dir) not in Windows. But that is misleading because "dir" is a builtin command but "ls" is not. The same holds for "del" (Windows) and "rm" (Linux).
That's exactly what shell=True is for - if you want a shell feature, you use the shell. What exactly would emulate_shell do? Would it simply do a $PATH or %PATH% search, but otherwise function as shell=False? Would it process redirection? Would it handle interpolations? I think not, from your description:
Perhaps it would be a good thing to emulate the builtin programs in python by some shell=True replacement (emulate_shell=True?) to match the normal user expectations without using the shell?
but it becomes difficult to draw the line. For instance, with emulate_shell=True, what would you do with all the sh/bash built-ins:
https://www.gnu.org/software/bash/manual/html_node/Bourne-Shell-Builtins.htm... https://www.gnu.org/software/bash/manual/html_node/Bash-Builtins.html
I'm thinking especially of the commands where bash has its own handling of something that could otherwise be found in $PATH, like pwd, time, and echo, but shells can do a lot of other things too.
When do you actually want to execute a shell built-in from Python but without using the shell itself? You give the example of ls/dir, but if that ever comes up in real-world code, I'd toss it out and recommend a cross-platform os.listdir or equivalent. There are plenty of times I've wanted a really quick way to redirect a standard stream from Python, but that isn't part of what you're recommending. Can you give a real-world example that would be improved by this?
I know this was just a side note in your original, but I'd like to hear more about what would make it useful.
No, I cannot. I just thought of a way to keep users from using "shell=True". I *think* they do it after they experience that "del" for instance is not found. They conclude "ah, I need the shell", which is not true. So whatever you come up with, the effect should be that people no longer use the shell. THATs what I want, after bad experience with non-escaped "^" in a regex, that caused some really weird result. -- Christian Tismer-Sperling :^) tismer@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam : GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023
Hi Chris,
On 07.01.18 18:18, Chris Angelico wrote:
Redirecting this part of the conversation to python-ideas.
On Mon, Jan 8, 2018 at 3:17 AM, Christian Tismer <tismer@stackless.com> wrote:
As a side note: In most cases where shell=True is found, people seem to need evaluation of the PATH variable. To my understanding,
from subprocess import call call(("ls",)) works in Linux, but (with dir) not in Windows. But that is misleading because "dir" is a builtin command but "ls" is not. The same holds for "del" (Windows) and "rm" (Linux). That's exactly what shell=True is for - if you want a shell feature, you use the shell. What exactly would emulate_shell do? Would it simply do a $PATH or %PATH% search, but otherwise function as shell=False? Would it process redirection? Would it handle interpolations? I think not, from your description:
Perhaps it would be a good thing to emulate the builtin programs in python by some shell=True replacement (emulate_shell=True?) to match the normal user expectations without using the shell? but it becomes difficult to draw the line. For instance, with emulate_shell=True, what would you do with all the sh/bash built-ins:
https://www.gnu.org/software/bash/manual/html_node/Bourne-Shell-Builtins.htm... https://www.gnu.org/software/bash/manual/html_node/Bash-Builtins.html
I'm thinking especially of the commands where bash has its own handling of something that could otherwise be found in $PATH, like pwd, time, and echo, but shells can do a lot of other things too.
When do you actually want to execute a shell built-in from Python but without using the shell itself? You give the example of ls/dir, but if that ever comes up in real-world code, I'd toss it out and recommend a cross-platform os.listdir or equivalent. There are plenty of times I've wanted a really quick way to redirect a standard stream from Python, but that isn't part of what you're recommending. Can you give a real-world example that would be improved by this?
I know this was just a side note in your original, but I'd like to hear more about what would make it useful.
No, I cannot. I just thought of a way to keep users from using "shell=True". I *think* they do it after they experience that "del" for instance is not found. They conclude "ah, I need the shell", which is not true. Even putting aside the fact this is pure conjecture, the kind of people who make decisions like this will find a zillion more ways to shoot
On 07.01.2018 22:32, Christian Tismer wrote: themselves in the foot. They don't need a cleaner syntax, they need to learn the basics of programming in a high-level language to understand how it's different from programming in the shell. In particular, why spawning a subprocess for something covered by a library function is a bad idea.
So whatever you come up with, the effect should be that people no longer use the shell. THATs what I want, after bad experience with non-escaped "^" in a regex, that caused some really weird result.
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Regards, Ivan
Reacting to:
No, I cannot. I just thought of a way to keep users from using "shell=True". I *think* they do it after they experience that "del" for instance is not found. They conclude "ah, I need the shell", which is not true.
Even putting aside the fact this is pure conjecture, the kind of people who make decisions like this will find a zillion more ways to shoot themselves in the foot. They don't need a cleaner syntax, they need to learn the basics of programming in a high-level language to understand how it's different from programming in the shell. In particular, why spawning a subprocess for something covered by a library function is a bad idea.
So whatever you come up with, the effect should be that people no longer use the shell. THATs what I want, after bad experience with non-escaped "^" in a regex, that caused some really weird result.
How about starting off with marking all use of "shell=True" as deprecated and then replacing the parameter with "risky_shell=True" or having no such parameter and adding "risky_" or "dangerous_" wrappers for all items that currently have the "shell=True" option. This would at least highlight that the developer is performing a risky operation, to me a part of the problem is that "shell=True" sounds innocuous so it is rarely picked up as a potential problem. I do quite like the idea of having a "with_path=True|False" option or maybe a "use_path=" that defaults to sys.path for all of the subprocess functions that would allow a little more control over the execution environment. -- Steve (Gadget) Barnes Any opinions in this message are my personal opinions and do not reflect those of my employer.
participants (4)
-
Chris Angelico -
Christian Tismer -
Ivan Pozdeev -
Steve Barnes