[Python-ideas] Better comand line version of python -c

[Andrew Barnert]

On Thursday, January 8, 2015 1:11 PM, Russell Stewart <Russell.S.Stewart at gmail.com> wrote:


>I am the original dev for pypi.python.org/pypi/pythonpy, which offers much of the same functionality as python -c, but in a more convenient form. I'd be really interested in figuring out whether the core python tools could support many of the use cases that pythonpy covers, so that users wouldn't have to install it as a separate tool. I don't have a concrete idea of what it would look like, but many of my users have said they would much prefer an officially supported tool. Perhaps someone on this mailing list can provide some insight into the available options.

This looks pretty cool to me—so I have lots of questions and comments. :)

The obvious problem is what to call it. `py` (well, `py.exe`) is already the name of the PEP 397 launcher tool (the thing that lets you use shbang lines on Windows), so this would need a new name.

Maybe it would be better as just a different flag or set of flags on `python` itself, like `--pprint`/`-p`? Of course then there's the problem of where to stick the flags for `-p` (for GNU-style long names this is pretty easy: `--print=filter,lines`, but obviously for something meant to be used for quick&dirty command-line usage you need short names as well…). And the "backport" to 3.4 (and 2.x) would act differently from the standard 3.5+ version. But it might be worth looking at anyway.


Or, of course, just make this a stdlib module, so it's just `python [PYTHON OPTIONS] -m YOUR_MODULENAME [YOUR MODULE OPTIONS]`, and you can alias that to whatever you want.

Is `-fx` a multi-character single-hyphen flag (in which case it's very weird to mix those with GNU-style long arguments in the same program, or is it a combination of `-f` and `-x`? And is the filter expression an argument to `-fx`, or does `-f` just change the interpretation of the argument?


Speaking of GNU-style long arguments, why `--i` instead of just `-i` (as a short name for `--ignore_exceptions` or something)

The documentation is pretty Python 2-specific: the majority of the examples use the `range` function and depend on it returning a list.


You seem to have built a non-trivial custom pretty-printer for this tool (e.g., to print lists row by row) as well as an auto-importer (e.g., to use `collections.Counter` without an `import collections`); maybe some or all of that should be separated out and made available to Python code somewhere in the stdlib, and then the script (or flag) could just use that function?

This seems to be pretty strongly list-based for no good reason. Why not print _any_ iterable line by line, make -l set `l = sys.stdin` instead of `l = list(sys.stdin)` (I realize that doesn't work for the specific example of `py -l 'l[::-1]'`, but `py -l reversed(l)` would work just as well in that case, and it's hard to think of other examples where there would even be a problem), etc.?

It might be nice to have an option to see the `repr` instead of the `str` of everything, to match what you'd see from the interactive interpreter.

While --si is cool, people already misuse `str.split` and `re.split` to try to parse CSV and similar input that has quotes or escapes; it might be nice to have a `--csv` mode to parse the input with `csv.reader` (and that could still take an optional delimiter, of course; passing other dialect flags to `reader` might be out of scope).

A columnar reading mode might also be nice, since that's one of those things that's novices have a hard time writing without statements.

An option for pretty JSON output instead of compact JSON output might be nice.

And maybe an option to read multi-line JSON input—not just a single JSON value (which is trivial with `json.loads(l)`, but, since JSON is self-delimiting, still a stream of them, without the requirement of one/line (e.g., by looping over `json.JSONDecoder.raw_decode`).

I assume you're evaluating the expression with `eval`. Are you passing it a custom locals and globals (so any internals of your script itself aren't available), or not? I could see disadvantages of both (the former may prevent some useful quick&dirty hacks; the latter could raise safety concerns), but either way, it's probably worth documenting.

>From the summary help, it looks like you can apply both `-x` and `-fx`. If you do that, do they run in a specific order, or in the order specified? For that matter, can you apply multiple mappings and/or filterings? (If not, it seems like that could be handy.)

It might be useful to have some kind of "daemon mode", where you start up an interpreter in the background and then pipe input to it. Then you can pipe multiple things to the same interpreter session, both for persistence, and to avoid the process startup cost on platforms where it matters (like Windows). Maybe with an optional timeout, where the daemon stops a few minutes after last use, so you don't have to remember to `py -d sys.exit(0)` or similar. And maybe the magic `_` should work in daemon mode, as it does in the interactive interpreter. (I could see writing some one-liner that takes 90 seconds to run, then realizing I wanted it in JSON format, so `py --daemon -jo _` would be handy to avoid re-running everything…)

It would be really cool if, when run under PowerShell, this could handle scriptlet input and output instead of plain text (so, e.g., you pass it an array of strings and it processes each string).

Being able to somehow pass command-line arguments through to `python` itself (assuming this isn't merged into the main interpreter, of course) might be handy. In particular, I could see `-u` being useful, but there might be others (including platform/implementation-specific `-X` options). Of course that would come for free with the `-m` interface, but since I'm not sure that's the best option...

Instead of just being able to ignore exceptions, it might be nice to enable one-line exception output (e.g., print just the type and message, no traceback, and to stdout instead of stderr).

With `-x`, an option to prefix each line of output with the corresponding line of input could be handy, similar to the `-v` option to commands like `cp`, `tar`, etc.

A way to feed input files in could be handy to allow the kind of one-liners people often fall back to perl for, although I'm not sure exactly what you'd want this to look like.

Some of these are probably way out of scope even for a third-party tool, much less for a built-in stdlib version, but I'm not sure exactly which, so I've just dumped everything on you to let you sort them out. :)