<div dir="ltr"><div>Andrew,</div><div><br></div><div>You've brought up many design points that I might have done differently if I were to start over. Things got locked in after the first few hundred users, but bringing things under the primary python command would free up a lot more room without backwards compatibility.</div><div><br></div><div>Heres some thoughts on a few things you said, assuming a brand new version without backwards compatibility would be the way forward:</div><div><br></div><div>Json handling:</div><div>I think I would remove this if I did it again. I originally found it quite useful, but the fact that it doesn't support multi-line json inputs makes it weird. Also, all the functionality can be accessed through the json module with python (e.g. py -l 'json.loads(l)').</div><div><br></div><div>Split input/output:</div><div>This one is actually super useful for parsing lines of text that you get from other unix commands (e.g. py --si 'special: ' -x x[1][:4] will grab the first 4 characters after the word special in any input file). It's also much easier to provide complete access to what people want to do with the single delimeter flag, so its much better than the --ji --jo flags in that respect.</div><div><br></div><div>-csv option:</div><div>I considered this at length, and originally the --si was motivated by commas. But ultimately --si is good for lots of things other than commas. A separate csv command seems nice, until you realize that there are too many options to specify in parameters, just like in the json case. It's better to let people parse csv docs with py -l 'csv.Reader(l) ... '. </div><div><br></div><div>Windows py.exe issue:</div><div>Yea, I'm aware of this. I have a pythonwpy for windows that is run with wpy, but no one seems to care about comand line tools on windows.</div><div><br></div><div>py vs python -m pythonpy ...:</div><div>Yes, using this with python -m is a very real option. People could simply alias py='python -m pythonpy' to avoid typing issues. This is probably the most straightforward distribution option. It would be great to keep the ipython style tab completion, but I'm not sure that its necessary.</div><div><br></div><div>py -fx flag:</div><div>Originally, I didn't want to name this -f because I didn't want the active variable x to be magically introduced. When you type -fx, its clear where the variable x comes from. This was probably a mistake, because everyone hates 2 character single-dash flags. Should be changed in the case of an overhaul. This command is also only marginally useful, due to the fact that grep already handles this use case so well, and because you can get the functionality with py -x 'x if foo(x) else None' just as well.</div><div><br></div><div>pythonpy should be Iterable based, not list-based:</div><div>I agree somewhat. The current dictionary functionality is weird, only printing the list of keys. Again, I don't want to change this for backwards compat reasons. This could be done a lot of different ways, and I think the current strategy is pretty good, but leaves a little bit to be desired in terms of simplicity. The -x and -l flags capture a lot of the list-printing that you would want to do, so its not clear you need to do list-printing with no arguments. Using a separate flag for list printing would be a good design consideration here. Keep in mind that the current motivation for all the list strangeness is so that pythonpy will play well with unix pipes, which it does. That is really important for some core usecases.</div><div><br></div><div>Daemon mode:</div><div>I hadn't thought of this one before. Maybe it could be done well, but daemon modes tend incur a high cost of mental overhead on the user. I've worked hard to keep the startup time within a factor of 2 from the python interpreter, so I do care a lot about that.</div><div><br></div><div>--i flag:</div><div>Again, the naming here is up for change. It's useful in a few cases, where python throws exceptions easily (e.g. py -x re.match(r"b", x).group(0)). But I haven't used it all that much, and it really could be removed.</div><div><br></div><div>So in general I like the python -m idea. Support for tab completion under this would be really nice. I've long thought that python -c and python -m should have ipython style tab completion anyways, so that one could type python -m Sim<tab> and get python -m SimpleHTTPServer. Another idea would be to try and get a shorter binary name distributed with python. It sounds superficial, but is ultimately quite important if python wants to compete with ruby, perl, and sed.</div><div><br></div><div>Best,</div><div>Russell</div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 8, 2015 at 2:35 PM, Andrew Barnert <span dir="ltr"><<a href="mailto:abarnert@yahoo.com" target="_blank">abarnert@yahoo.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span>On Thursday, January 8, 2015 1:11 PM, Russell Stewart <<a href="mailto:Russell.S.Stewart@gmail.com" target="_blank">Russell.S.Stewart@gmail.com</a>> wrote:<br>
<br>
<br>
>I am the original dev for <a href="http://pypi.python.org/pypi/pythonpy" target="_blank">pypi.python.org/pypi/pythonpy</a>, which offers much of the same functionality as python -c, but in a more convenient form. I'd be really interested in figuring out whether the core python tools could support many of the use cases that pythonpy covers, so that users wouldn't have to install it as a separate tool. I don't have a concrete idea of what it would look like, but many of my users have said they would much prefer an officially supported tool. Perhaps someone on this mailing list can provide some insight into the available options.<br>
<br>
</span>This looks pretty cool to me—so I have lots of questions and comments. :)<br>
<br>
The obvious problem is what to call it. `py` (well, `py.exe`) is already the name of the PEP 397 launcher tool (the thing that lets you use shbang lines on Windows), so this would need a new name.<br>
<br>
Maybe it would be better as just a different flag or set of flags on `python` itself, like `--pprint`/`-p`? Of course then there's the problem of where to stick the flags for `-p` (for GNU-style long names this is pretty easy: `--print=filter,lines`, but obviously for something meant to be used for quick&dirty command-line usage you need short names as well…). And the "backport" to 3.4 (and 2.x) would act differently from the standard 3.5+ version. But it might be worth looking at anyway.<br>
<br>
<br>
Or, of course, just make this a stdlib module, so it's just `python [PYTHON OPTIONS] -m YOUR_MODULENAME [YOUR MODULE OPTIONS]`, and you can alias that to whatever you want.<br>
<br>
Is `-fx` a multi-character single-hyphen flag (in which case it's very weird to mix those with GNU-style long arguments in the same program, or is it a combination of `-f` and `-x`? And is the filter expression an argument to `-fx`, or does `-f` just change the interpretation of the argument?<br>
<br>
<br>
Speaking of GNU-style long arguments, why `--i` instead of just `-i` (as a short name for `--ignore_exceptions` or something)<br>
<br>
The documentation is pretty Python 2-specific: the majority of the examples use the `range` function and depend on it returning a list.<br>
<br>
<br>
You seem to have built a non-trivial custom pretty-printer for this tool (e.g., to print lists row by row) as well as an auto-importer (e.g., to use `collections.Counter` without an `import collections`); maybe some or all of that should be separated out and made available to Python code somewhere in the stdlib, and then the script (or flag) could just use that function?<br>
<br>
This seems to be pretty strongly list-based for no good reason. Why not print _any_ iterable line by line, make -l set `l = sys.stdin` instead of `l = list(sys.stdin)` (I realize that doesn't work for the specific example of `py -l 'l[::-1]'`, but `py -l reversed(l)` would work just as well in that case, and it's hard to think of other examples where there would even be a problem), etc.?<br>
<br>
It might be nice to have an option to see the `repr` instead of the `str` of everything, to match what you'd see from the interactive interpreter.<br>
<br>
While --si is cool, people already misuse `str.split` and `re.split` to try to parse CSV and similar input that has quotes or escapes; it might be nice to have a `--csv` mode to parse the input with `csv.reader` (and that could still take an optional delimiter, of course; passing other dialect flags to `reader` might be out of scope).<br>
<br>
A columnar reading mode might also be nice, since that's one of those things that's novices have a hard time writing without statements.<br>
<br>
An option for pretty JSON output instead of compact JSON output might be nice.<br>
<br>
And maybe an option to read multi-line JSON input—not just a single JSON value (which is trivial with `json.loads(l)`, but, since JSON is self-delimiting, still a stream of them, without the requirement of one/line (e.g., by looping over `json.JSONDecoder.raw_decode`).<br>
<br>
I assume you're evaluating the expression with `eval`. Are you passing it a custom locals and globals (so any internals of your script itself aren't available), or not? I could see disadvantages of both (the former may prevent some useful quick&dirty hacks; the latter could raise safety concerns), but either way, it's probably worth documenting.<br>
<br>
>From the summary help, it looks like you can apply both `-x` and `-fx`. If you do that, do they run in a specific order, or in the order specified? For that matter, can you apply multiple mappings and/or filterings? (If not, it seems like that could be handy.)<br>
<br>
It might be useful to have some kind of "daemon mode", where you start up an interpreter in the background and then pipe input to it. Then you can pipe multiple things to the same interpreter session, both for persistence, and to avoid the process startup cost on platforms where it matters (like Windows). Maybe with an optional timeout, where the daemon stops a few minutes after last use, so you don't have to remember to `py -d sys.exit(0)` or similar. And maybe the magic `_` should work in daemon mode, as it does in the interactive interpreter. (I could see writing some one-liner that takes 90 seconds to run, then realizing I wanted it in JSON format, so `py --daemon -jo _` would be handy to avoid re-running everything…)<br>
<br>
It would be really cool if, when run under PowerShell, this could handle scriptlet input and output instead of plain text (so, e.g., you pass it an array of strings and it processes each string).<br>
<br>
Being able to somehow pass command-line arguments through to `python` itself (assuming this isn't merged into the main interpreter, of course) might be handy. In particular, I could see `-u` being useful, but there might be others (including platform/implementation-specific `-X` options). Of course that would come for free with the `-m` interface, but since I'm not sure that's the best option...<br>
<br>
Instead of just being able to ignore exceptions, it might be nice to enable one-line exception output (e.g., print just the type and message, no traceback, and to stdout instead of stderr).<br>
<br>
With `-x`, an option to prefix each line of output with the corresponding line of input could be handy, similar to the `-v` option to commands like `cp`, `tar`, etc.<br>
<br>
A way to feed input files in could be handy to allow the kind of one-liners people often fall back to perl for, although I'm not sure exactly what you'd want this to look like.<br>
<br>
Some of these are probably way out of scope even for a third-party tool, much less for a built-in stdlib version, but I'm not sure exactly which, so I've just dumped everything on you to let you sort them out. :)<br>
</blockquote></div><br></div></div>