unix filter, one liners, awk
Is there a reason, why not to make python useful for practical one liners to replace perl and awk? There is page Powerful Python One-Liners, https://wiki.python.org/moin/Powerful%20Python%20One-Liners. But almost none of them handles files, behaves like unix filter or is ugly – must import modules, etc. It seems that the command line options like -a, -n, -p, -F are still free. :-) https://docs.python.org/3/using/cmdline.html Why not use them almost the same way as in Perl? https://perldoc.perl.org/perlrun#-n https://perldoc.perl.org/perlrun#-p https://perldoc.perl.org/perlrun#-a https://perldoc.perl.org/perlrun#-Fpattern E.g. -n would be almost equivalent to import sys,os,re from fileinput import * for line in input(): <-c code comes here> close() -p will print(line) as last command of the for cycle. Why to learn Perl/awk/datamash/mlr/…, for “one line like” tasks? Thank you in advance Hans PS: https://blogs.oracle.com/linux/the-top-10-tricks-of-perl-one-liners-v2 https://gist.github.com/joyrexus/7328094 https://stackoverflow.com/questions/1589994/how-do-i-write-a-unix-filter-in-... Why not to have an elegant (one line) answer in python for questions like this? https://stackoverflow.com/questions/40708370/drop-duplicates-and-keep-first-...
IIRC I created the fileinput module to help with this, but I don't know how much use it finds. On Thu, Nov 5, 2020 at 10:05 AM Hans Ginzel <hans@matfyz.cz> wrote:
Is there a reason, why not to make python useful for practical one liners to replace perl and awk? There is page Powerful Python One-Liners, https://wiki.python.org/moin/Powerful%20Python%20One-Liners. But almost none of them handles files, behaves like unix filter or is ugly – must import modules, etc.
It seems that the command line options like -a, -n, -p, -F are still free. :-) https://docs.python.org/3/using/cmdline.html
Why not use them almost the same way as in Perl? https://perldoc.perl.org/perlrun#-n https://perldoc.perl.org/perlrun#-p https://perldoc.perl.org/perlrun#-a https://perldoc.perl.org/perlrun#-Fpattern
E.g. -n would be almost equivalent to
import sys,os,re from fileinput import * for line in input(): <-c code comes here> close()
-p will print(line) as last command of the for cycle.
Why to learn Perl/awk/datamash/mlr/…, for “one line like” tasks?
Thank you in advance Hans
PS: https://blogs.oracle.com/linux/the-top-10-tricks-of-perl-one-liners-v2 https://gist.github.com/joyrexus/7328094
https://stackoverflow.com/questions/1589994/how-do-i-write-a-unix-filter-in-... Why not to have an elegant (one line) answer in python for questions like this?
https://stackoverflow.com/questions/40708370/drop-duplicates-and-keep-first-... _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/IT5MXI... Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
Hello, On Thu, 5 Nov 2020 18:46:53 +0100 Hans Ginzel <hans@matfyz.cz> wrote:
Is there a reason, why not to make python useful for practical one liners to replace perl and awk? There is page Powerful Python One-Liners, https://wiki.python.org/moin/Powerful%20Python%20One-Liners. But almost none of them handles files, behaves like unix filter or is ugly – must import modules, etc.
It seems that the command line options like -a, -n, -p, -F are still free. :-) https://docs.python.org/3/using/cmdline.html
Thankfully, command-line option "-m" is there for ages. And you're looking for python3 -m <my_favorite_DSL> <any_arguments> Where <my_favorite_DSL> was put on https://pypi.org/ by you or similarly-minded people, and installed with: pip3 install --user <my_favorite_DSL> before running the previous command (needs to be done only once).
Why not use them almost the same way as in Perl? https://perldoc.perl.org/perlrun#-n https://perldoc.perl.org/perlrun#-p https://perldoc.perl.org/perlrun#-a https://perldoc.perl.org/perlrun#-Fpattern
E.g. -n would be almost equivalent to
import sys,os,re from fileinput import * for line in input(): <-c code comes here> close()
-p will print(line) as last command of the for cycle.
Why to learn Perl/awk/datamash/mlr/…, for “one line like” tasks?
Thank you in advance Hans
PS: https://blogs.oracle.com/linux/the-top-10-tricks-of-perl-one-liners-v2 https://gist.github.com/joyrexus/7328094 https://stackoverflow.com/questions/1589994/how-do-i-write-a-unix-filter-in-... Why not to have an elegant (one line) answer in python for questions like this? https://stackoverflow.com/questions/40708370/drop-duplicates-and-keep-first-...
[] -- Best regards, Paul mailto:pmiscml@gmail.com
On Thu, Nov 05, 2020 at 09:41:45PM +0300, Paul Sokolovsky wrote:
Hello,
On Thu, 5 Nov 2020 18:46:53 +0100 Hans Ginzel <hans@matfyz.cz> wrote:
Is there a reason, why not to make python useful for practical one liners to replace perl and awk? There is page Powerful Python One-Liners, https://wiki.python.org/moin/Powerful%20Python%20One-Liners. But almost none of them handles files, behaves like unix filter or is ugly – must import modules, etc.
It seems that the command line options like -a, -n, -p, -F are still free. :-) https://docs.python.org/3/using/cmdline.html
I wish I had a dollar for every time I called `python -c ...` from the command line and forgot to explicitly call print. The interactive interpreter's automatic printing of expressions is extremely user-friendly, in comparison `python -c` much less so. Likewise, the need to handle imports, explicit loops with newlines and indents etc makes Python much harder to use from the command line, and so much less useful. Many a time I've tried to run some quick command, couldn't get it to work.
Thankfully, command-line option "-m" is there for ages. And you're looking for
python3 -m <my_favorite_DSL> <any_arguments>
I don't think that is what Hans is asking for. I'm pretty sure Hans is not looking for a pre-existing script to provide arguments to. He is looking to write: python <some invocation> <python code> So more like `python -c ...` than `python -m ...`, only with some boilerplate needed for file handling performed automatically. Hans even explicitly refers to `-c` in his post. Perhaps that invocation will involve `-m`, like pydoc. But I think that there is a good case for mapping such a command line helper to a single option letter, even if that ultimately ends up being managed by a script rather than built into the interpreter.
Where <my_favorite_DSL> was put on https://pypi.org/ by you or similarly-minded people,
Do you have an *actual* "my_favorite_DSL" in mind? If you don't have a well-known package in mind, how do you expect Python users in general to know which "my_favorite_DSL" will solve this problem? It might turn out that the right solution here will involve calling a script, like we call `-m unittest` or `-m doctest` etc. Maybe the best place for that is PyPI, but we shouldn't just dismiss Hans' suggestion with "just get it from PyPI".
and installed with:
pip3 install --user <my_favorite_DSL>
before running the previous command (needs to be done only once).
Needs to be done *once per user*, *per computer*. And in many environments, that would have to be followed up by applying for unemployment benefits, assuming that you are still eligible for benefits after being fired with cause. "Just use pip" is a truly privileged position to take, one which assumes that you have the ability and legal right to install third party software on the machine you are using, with unfettered access to PyPI. In the FOSS community, it is easy to forget that this is not the norm for many people. They use school laptops or corporate machines with restrictive policies and permissions, or they have regulatory laws that require long complex audits of any software before it can be installed. Or they are sys admins who are working on production servers and they trust their own command line skills over unknown third party libraries they download off the internet. I would be very interested in exploring what it takes to make it easier to use Python from the command line. Maybe Python's enforced indentation will make this a losing proposition, but I don't think we should be so dismissive of improving the command line functionality. -- Steve
Try https://github.com/ksamuel/Pyped On Thu, Nov 5, 2020 at 8:03 PM Hans Ginzel <hans@matfyz.cz> wrote:
Is there a reason, why not to make python useful for practical one liners to replace perl and awk? There is page Powerful Python One-Liners, https://wiki.python.org/moin/Powerful%20Python%20One-Liners. But almost none of them handles files, behaves like unix filter or is ugly – must import modules, etc.
It seems that the command line options like -a, -n, -p, -F are still free. :-) https://docs.python.org/3/using/cmdline.html
Why not use them almost the same way as in Perl? https://perldoc.perl.org/perlrun#-n https://perldoc.perl.org/perlrun#-p https://perldoc.perl.org/perlrun#-a https://perldoc.perl.org/perlrun#-Fpattern
E.g. -n would be almost equivalent to
import sys,os,re from fileinput import * for line in input(): <-c code comes here> close()
-p will print(line) as last command of the for cycle.
Why to learn Perl/awk/datamash/mlr/…, for “one line like” tasks?
Thank you in advance Hans
PS: https://blogs.oracle.com/linux/the-top-10-tricks-of-perl-one-liners-v2 https://gist.github.com/joyrexus/7328094
https://stackoverflow.com/questions/1589994/how-do-i-write-a-unix-filter-in-... Why not to have an elegant (one line) answer in python for questions like this?
https://stackoverflow.com/questions/40708370/drop-duplicates-and-keep-first-... _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/IT5MXI... Code of Conduct: http://python.org/psf/codeofconduct/
Alex Hall writes:
Or perhaps https://xon.sh, and have Python be the native syntax of the shell. (Unfortunately if I read correctly xonsh is based on Python 3.5, so no walrus operator and no f-strings yet.) On Thu, Nov 5, 2020 at 8:03 PM Hans Ginzel <hans@matfyz.cz> wrote:
Is there a reason, why not to make python useful for practical one liners to replace perl and awk? [...] Why to learn Perl/awk/datamash/mlr/…, for “one line like” tasks?
I'm really not clear on the concept here, though. My problem with the whole idea of Python one-liners is its modular architecture and syntax. It doesn't lend itself to one-liners as far as I can see. I guess you deal with significant whitespace by defining "line" as "logical line" and passing python a string formatted as a program would be, shells can handle that. But most work in Python involves function calls, which involve parentheses, so that on the command line you'll have to quote them, which means you have to keep track of Python and shell quoting semantics simultaneously if you use any literal strings. The perlrun manpage say "There is no general solution to all of this. It's just a mess." :-) As for function calls, even a simple regex filter ("grep") using -p will require if not re.search('some regexp', line): continue Sure, you could provide a command-line option for that; I suspect it's common enough to justify one more option if you're already adding four. :-) But there are infinite variations on the theme. Anyway, in most cases you have to call functions, and quite likely import modules that provide them. I guess that your initial list of os, re, sys covers most of what Perl provides natively, so maybe imports are not such a big problem. Although requiring the "os." (and "os.path."!) prefix is annoying -- I'm not sure it's a good idea to "from os import *", etc. It would help if you provided a variety of "<-c code comes here>" examples for your proposed "python -n -c ..." invocations to show that these extensions would be frequently useful. Eg, I think I'd type cat file | python -p "if not re.search('some regexp', line): continue" at most once before asking a friend if there's a command-line utility that takes a regexp and filters linewise. Also, given your example for -n, I wonder if you couldn't get most of the effect you want with modules, like this: python -m perln "<-c code comes here>" or even python -m oneline -n "<-c code comes here>" Don't ask me how to implement those; I'm pretty sure it's quite possible, but it's over my head. I second most of what Steve D'Aprano wrote, except that I think that "-m oneline -n" would be a good way to experiment (and I suspect something like this is what Alex had in mind when he wrote "DSL"). Steve
Am 06.11.20 um 08:15 schrieb Stephen J. Turnbull:
Alex Hall writes:
Or perhaps https://xon.sh, and have Python be the native syntax of the shell. (Unfortunately if I read correctly xonsh is based on Python 3.5, so no walrus operator and no f-strings yet.)
It says 3.5+. xonsh happily works with Python 3.9. ;)
Here's my take on a Python one liner tool: https://github.com/hauntsaninja/pyp It handles automatic intelligent printing, importing and more. It explicitly aims to feel very much like writing Python, as opposed to a DSL. In fact, it can even generate a runnable Python script to show you exactly what it's doing. README includes comparisons to Pyped, xonsh and others — there's a long history of people wanting more Python in their terminal :-) On Thu, 5 Nov 2020 at 23:35, Mike Müller <mmueller@python-academy.de> wrote:
Am 06.11.20 um 08:15 schrieb Stephen J. Turnbull:
Alex Hall writes:
Or perhaps https://xon.sh, and have Python be the native syntax of the shell. (Unfortunately if I read correctly xonsh is based on Python 3.5, so no walrus operator and no f-strings yet.)
It says 3.5+. xonsh happily works with Python 3.9. ;) _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/WCIOPG... Code of Conduct: http://python.org/psf/codeofconduct/
participants (8)
-
Alex Hall -
Guido van Rossum -
Hans Ginzel -
Mike Müller -
Paul Sokolovsky -
Shantanu Jain -
Stephen J. Turnbull -
Steven D'Aprano