synatx sugar for quickly run shell command and return stdout of shell command as string result
Currently, when what to execute external command, either os.system() or subprocess functions should be invoked. However, it is not so convenient, it would be nice to use "``" symbol to brace the shell command inside. like: stdout_result_str = `cat file.txt | sort | grep hello` Its is enhanced feature which combine both subprocess and os.system Because, subprocess can return output of file, but does not support pipe in command. While os.system can apply pipe in command but doesn't support return stdout result. This can be very convenient to make python available for some build script which ruby now do well like homebrew ... simply `echo hello | wc -l` while only execute command and the result will just like other unassigned expression, the result will be finally discarded. There won't be big difficulties to implement.
subprocess.run(args, capture_output=True, check=True, text=True, encoding="utf8").stdout ? On Thu, 26 Aug 2021, 13:55 Evan Greenup via Python-ideas, < python-ideas@python.org> wrote:
Currently, when what to execute external command, either os.system() or subprocess functions should be invoked.
However, it is not so convenient, it would be nice to use "``" symbol to brace the shell command inside.
like:
stdout_result_str = `cat file.txt | sort | grep hello`
Its is enhanced feature which combine both subprocess and os.system
Because, subprocess can return output of file, but does not support pipe in command.
While os.system can apply pipe in command but doesn't support return stdout result.
This can be very convenient to make python available for some build script which ruby now do well like homebrew ...
simply
`echo hello | wc -l` while only execute command and the result will just like other unassigned expression, the result will be finally discarded.
There won't be big difficulties to implement. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/5KKHG6... Code of Conduct: http://python.org/psf/codeofconduct/
There is a Python library called plumbum that provides a shell-like DSL for executing shell commands: https://plumbum.readthedocs.io/en/latest/
There is a Python library called plumbum that provides a shell-like DSL for executing shell commands:
Also xonch: https://xon.sh Python itself is purposely not designed to provide quick and easy shell access. -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
And also IPython: https://ipython.readthedocs.io/en/stable/interactive/shell.html e.g. ls_lines = !ls -l
On Thu, Aug 26, 2021, at 08:50, Evan Greenup via Python-ideas wrote:
Currently, when what to execute external command, either os.system() or subprocess functions should be invoked.
However, it is not so convenient, it would be nice to use "``" symbol to brace the shell command inside.
like:
stdout_result_str = `cat file.txt | sort | grep hello`
I don't think this is an important enough feature to be worth using up one of the few free ascii characters available for new syntax. However, it might be useful to have a helper function to build pipelines without the shell, so you could take the list [['cat', 'file.txt'], ['sort'], ['grep', 'hello']] and get this result. Like, currently, to set up this pipeline (putting aside the useless use of cat), you have to do the following: p1 = subprocess.Popen(['cat', 'file.txt'], stdout=PIPE) p2 = subprocess.Popen(['sort'], stdout=PIPE, stdin=p1.stdout) result = subprocess.check_output(['grep', 'hello'], stdin=p2.stdout) [there are nuances about closing the intermediate pipes in the parent process that I've left out of this simple example] something like subprocess.check_output_pipelined(['cat', 'file.txt'], ['sort'], ['grep', 'hello']) might be useful.
Its is enhanced feature which combine both subprocess and os.system
To do this *with* a shell, you can use os.popen and read the resulting file stream. But there are good reasons, I think, to want to be able to do this without a shell.
On Thu, Aug 26, 2021 at 5:53 AM Evan Greenup via Python-ideas < python-ideas@python.org> wrote:
Currently, when what to execute external command, either os.system() or subprocess functions should be invoked.
However, it is not so convenient, it would be nice to use "``" symbol to brace the shell command inside.
like:
stdout_result_str = `cat file.txt | sort | grep hello`
Its is enhanced feature which combine both subprocess and os.system
Because, subprocess can return output of file, but does not support pipe in command.
subprocess.run('cat file.txt | sort |grep hello', shell=True capture_output=True, check=True, text=True, encoding="utf8").stdout you can use any piping or redirection you like. However, I do like the idea of run() having the ability to easily chain commands without the need to pull in the shell. Is this too magical? result = run('cat file.txt') | run('sort) | run('grep hello', capture_output=True, text=True).stdout I'm sure there are other approaches. We don't often have to worry about file handles in python, and it would be nice to remove this worry from subprocess pipes even more than it already is.
Is this too magical? result = run('cat file.txt') | run('sort) | run('grep hello', capture_output=True, text=True).stdout
Interesting idea, especially overloading the union/pipe operator (|). I like it a lot. It reminds me of pathlib.Path (which is a wonderful tool), with its slash operator overload. Python is supposed to be easy to read and understand, and anything that makes complicated system stuff less messy is welcome, I think. Another idea (which I am less in favor of) is a more functional approach, like: run('grep hello', run('sort', run('cat file.txt'))) In this version, it looks more like Python functions than pipes, where the last (optional) argument is what's piped in. This makes more sense in a functional context and may eliminate the need for capture_output=True, text=True, .stdout, and similar idioms. Like I said, I'm not as much in favor of this, though, because it doesn't resemble an actual call as much. However, it's still an idea. It might be helpful to point out that Jeremiah's version would have to be implemented as a class because of the union operator overload. On Thu, Aug 26, 2021, 8:44 PM Jeremiah Paige <ucodery@gmail.com> wrote:
On Thu, Aug 26, 2021 at 5:53 AM Evan Greenup via Python-ideas < python-ideas@python.org> wrote:
Currently, when what to execute external command, either os.system() or subprocess functions should be invoked.
However, it is not so convenient, it would be nice to use "``" symbol to brace the shell command inside.
like:
stdout_result_str = `cat file.txt | sort | grep hello`
Its is enhanced feature which combine both subprocess and os.system
Because, subprocess can return output of file, but does not support pipe in command.
subprocess.run('cat file.txt | sort |grep hello', shell=True capture_output=True, check=True, text=True, encoding="utf8").stdout you can use any piping or redirection you like.
However, I do like the idea of run() having the ability to easily chain commands without the need to pull in the shell. Is this too magical? result = run('cat file.txt') | run('sort) | run('grep hello', capture_output=True, text=True).stdout I'm sure there are other approaches. We don't often have to worry about file handles in python, and it would be nice to remove this worry from subprocess pipes even more than it already is. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/TXUDW3... Code of Conduct: http://python.org/psf/codeofconduct/
On Fri, Aug 27, 2021 at 1:15 PM Finn Mason <finnjavier08@gmail.com> wrote:
Is this too magical? result = run('cat file.txt') | run('sort) | run('grep hello', capture_output=True, text=True).stdout
Interesting idea, especially overloading the union/pipe operator (|). I like it a lot. It reminds me of pathlib.Path (which is a wonderful tool), with its slash operator overload. Python is supposed to be easy to read and understand, and anything that makes complicated system stuff less messy is welcome, I think.
It's perfectly legal to do this sort of thing, but it doesn't really belong in the stdlib (more on that later).
Another idea (which I am less in favor of) is a more functional approach, like:
run('grep hello', run('sort', run('cat file.txt')))
That's a bit more viable. I'd be inclined to have a keyword argument here, for instance: run('grep hello', stdin=run('sort', sttdin=run('cat file.txt'))) but I agree, it makes good sense to have an argument able to take a variety of data types, including another program's output. But Python isn't a language that needs everything to be done with command invocations and pipes. If you're doing long pipelines like this, there's probably something wrong. Why use external programs to do things that Python can do far more viably itself? with open('file.txt') as f: result = [l.strip("\r\n") for l in sorted(f) if "hello" in l] Trying to make a more Pythonic syntax for an operation that Python does better in a completely different way is a bit of a half-hearted solution. I'd recommend either sticking to a full "run the shell and hand it this command line", or going the whole way and doing the job in Python. ChrisA
But Python isn't a language that needs everything to be done with command invocations and pipes. If you're doing long pipelines like this, there's probably something wrong. Why use external programs to do things that Python can do far more viably itself?
That's a really good point. I think the main use case is scripts run from the command line, like pip or virtualenv, for example. It's uncommon enough that's it's definitely not necessary. The "grep hello | sort | cat file.txt" example is just that: an example. I'm no expert on system calls from within Python. Does it open a new process that Python doesn't have to wait on? If so, it might be more viable. On Thu, Aug 26, 2021, 9:22 PM Chris Angelico <rosuav@gmail.com> wrote:
On Fri, Aug 27, 2021 at 1:15 PM Finn Mason <finnjavier08@gmail.com> wrote:
Is this too magical? result = run('cat file.txt') | run('sort) | run('grep hello',
capture_output=True, text=True).stdout
Interesting idea, especially overloading the union/pipe operator (|). I
like it a lot. It reminds me of pathlib.Path (which is a wonderful tool), with its slash operator overload.
Python is supposed to be easy to read and understand, and anything that makes complicated system stuff less messy is welcome, I think.
It's perfectly legal to do this sort of thing, but it doesn't really belong in the stdlib (more on that later).
Another idea (which I am less in favor of) is a more functional approach, like:
run('grep hello', run('sort', run('cat file.txt')))
That's a bit more viable. I'd be inclined to have a keyword argument here, for instance:
run('grep hello', stdin=run('sort', sttdin=run('cat file.txt')))
but I agree, it makes good sense to have an argument able to take a variety of data types, including another program's output.
But Python isn't a language that needs everything to be done with command invocations and pipes. If you're doing long pipelines like this, there's probably something wrong. Why use external programs to do things that Python can do far more viably itself?
with open('file.txt') as f: result = [l.strip("\r\n") for l in sorted(f) if "hello" in l]
Trying to make a more Pythonic syntax for an operation that Python does better in a completely different way is a bit of a half-hearted solution. I'd recommend either sticking to a full "run the shell and hand it this command line", or going the whole way and doing the job in Python.
ChrisA _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/WUZH3G... Code of Conduct: http://python.org/psf/codeofconduct/
On Thu, Aug 26, 2021 at 09:15:21PM -0600, Finn Mason <finnjavier08@gmail.com> wrote:
Is this too magical? result = run('cat file.txt') | run('sort) | run('grep hello', capture_output=True, text=True).stdout
Interesting idea, especially overloading the union/pipe operator (|). I like it a lot. It reminds me of pathlib.Path (which is a wonderful tool), with its slash operator overload.
The idea is quite standard now. See https://pypi.org/project/shell-pipes/ https://pypi.org/search/?q=shell+pipes Oleg. -- Oleg Broytman https://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.
participants (11)
-
Angus Hollands -
Chris Angelico -
Christopher Barker -
Evan Greenup -
Finn Mason -
Jeremiah Paige -
Oleg Broytman -
Random832 -
Rob Cliffe -
Thomas Grainger -
tim.hoffmann＠mailbox.org