[Python-ideas] Better comand line version of python -c

[Andrew Barnert]

On Thursday, January 8, 2015 10:10 PM, Ed Kellett <edk141 at gmail.com> wrote:

>On a completely different note, do you have any specific thoughts about how the daemon mode is going to work? I considered it for spy, but getting it exactly right is non-trivial:

I was having a hard time trying to explain it, so after my third rewrite, I decided to instead hack up a (very) quick&dirty eval daemon (https://github.com/abarnert/pyd).

The daemon part is basically invisible. If a daemon is already running, it's used; if not, a new one is started; to kill an existing daemon, just run with no expression. (Actually, it's not quite invisible; I added in some lines to print the daemon's pid when launched and to tell you whether it was running when killed, for debugging purposes. But those could be removed.)

Your expression gets passed to `eval`, your input gets sent to the expression's stdin line by line, its stdout and stderr get sent to your stdout line by line, and then the result of your expression gets stored in `_` for next time, and printed.

For example:

$ echo | ./pyd 2+3
Started daemon on 8089
5
$ echo | ./pyd _+3
8
$ echo -e 'abc\ndef\n' | ./pyd 'input()'
abc
$ echo | ./pyd _+3
TypeError: unsupported operand type(s) for +: 'int' and 'str'
$ echo | ./pyd '_+"3"'
abc3
$ ./pyd
Daemon killed
$ ./pyd
Daemon was not running
$ echo | ./pyd _+3
NameError: name '_' is not defined
$ echo | ./pyd 2+3
5
$ echo | ./pyd _+3
8

You're probably wondering what's with the `echo`. Proxying `sys.stdin` line by line to a socket in both terminal and non-terminal mode on Mac, Linux, and Cygwin has some annoying problems, and the workaround takes more than a couple lines, so I made the code loop over `input` instead, which means it ends with a blank line rather than EOF, which sucks as a UI but it's fine for testing the functionality.

I haven't thought through all of the issues with in what senses this is and isn't a daemon; I just closed stdin/out/err and called `setsid`. I also didn't include a timeout argument; the daemon just runs until you kill it. The `eval` is done in the daemon's own environment. There's probably lots of missing error handling. It's only tested on Mac and Fedora, and definitely won't work on (native) Windows. Of course it doesn't do any of the fancy stuff pythonpy or spy does; it just calls eval. It may not work with very long expressions or very long input lines or lots of separate output writes, and it definitely doesn't work if the output is more than 4K. But all of those things are doable in a real program if you have more than 15 minutes to write it. :)

>- Multiple users on the same system should definitely work

That's easy; my hack creates the socket under `$HOME`. A real script would hopefully use somewhere XDG/Mac-compliant if relevant. Or you could encode the uid into the socket name and store it in `$TMP` if you prefer.

>- Different pythons or different virtualenvs under the same user should work

I apologize that I haven't looked at spy yet, but with pythonpy it looks like you have to have a separate script for each Python, which means all you need to do it add the script's name (`basename(argv[0])`) to the socket name.

If you want to do something like `python -m pythonpy 2+3`, then you'll probably want the full path to `sys.executable` instead (which means you need to encode it in some way to deal with slashes, of course).

>- Where should the daemon's stdout go?

Back over the socket.

(There shouldn't be any stdout between connections. If there is, it'll go nowhere.)

>- If `_` is present, what will happen when you inevitably end up working on two things at once in different terminals?

Whoever runs last (note that the hack doesn't allow concurrent connections; I'm not sure if you'd want to or not) overwrites `_`.

Personally, the use cases I'm envisioning for `_` are accessing it immediately after the last command--typically because I'm an idiot and forgot a step and don't want to have to re-run the previous step just to add another one (the same reason I usually use `_` in the interactive interpreter). So, I think this is fine.

But if you want separate sessions, that's easy. Just encode the sid in the socket name, so you have one daemon per session.

If you want to get fancy, you could instead use an environment variable that's set to the sid (or just to a random UUID or whatever) if not present; that way by default each terminal gets its own daemon, but if you want to share a daemon between sessions all you have to do is set the env variable manually to match.

You might also want to consider whether the daemon should die when the controlling shell/session/terminal dies, if you have a separate daemon per terminal.

>- If this requires any manual help from the user, we've lost. It's not convenient enough

It shouldn't. If you want to provide options to let the user get fancy you can (like the last paragraph above), but I don't think any such options are necessary. The only options I might provide are to use the daemon or not, and an idle timeout to kill the daemon.