Syntax to import modules before running command from the command line
Currently invoking `python -c "some;separated;set of commands;"` will, if you need to use any library functions, require one or more import somelib; sections in the execution string. This results in rather complex "one liners". On the other hand `python -m somelib` will load somelib and attempt to execute its `__main__()` or give an error if there isn't one. What I would like to suggest is a mechanism to pre-load libraries before evaluating the -c option as this would allow the use of code from libraries that don't have a `__main__` function, or those that do but it doesn't do what you want. Since -m for module is already taken I would suggest one of: -p for pre-load module -M for load module without attempting to execute `module.__main__()` and without defining "__main__" in the load context or -l for library with the last two having the advantage of appearing next to -m in the --help output. This would change, (for a trivial example): `python -c"import numpy;print(numpy.pi);"` to: `python -M numpy -c"print(numpy.pi);"` -- Steve (Gadget) Barnes Any opinions in this message are my personal opinions and do not reflect those of my employer.
Could it just check if -c and -m are both set? That way there'd be no need for -p or -M. (I have an -m <module> switch in pyline which does exactly this. It makes copying and pasting less convenient; but does save having to type 'import module;’ for one liners) On Friday, January 5, 2018, Steve Barnes <gadgetsteve@live.co.uk> wrote:
On 5 January 2018 at 16:28, Steve Barnes <gadgetsteve@live.co.uk> wrote:
That's not quite how the -m switch works, but that doesn't affect your core point: "-m somemodule" terminates the option list and defines what will be run as `__main__`, so you can't compose it with other options (like `-c`).
This could be an interesting alternative to the old "python -C" idea in https://bugs.python.org/issue14803 (which suggests adding an alternative to "-c" that runs the given code *before* running `__main__`, rather than running it *as* `__main__`). To summarise some of the use cases from that discussion: * setting `__main__.__requires__` to change how `import pkg_resources` selects default package versions * reducing the proliferation of `-X` options to enable modules like tracemalloc and faulthandler * providing a way to enable code coverage as early as possible on startup (and have the setting be inherited by subprocesses) The main downside I'd see compared to that `python -C` and PYTHONRUNFIRST idea is that it might encourage the definition of modules with side-effects in order to use it to reconfigure the interpreter at startup. That concern could potentially be mitigated if we did both: * -M/PYTHONIMPORTFIRST: implicit imports in __main__ on startup * -C/PYTHONRUNFIRST: code to run in __main__ on startup (after the implicit imports) However, the issue then is that "python -M numpy" would just be a less flexible alternative to a command like "python -C 'import numpy as np'". Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 5 January 2018 at 08:12, Nick Coghlan <ncoghlan@gmail.com> wrote:
For quick one-liners don't underestimate the value of avoiding punctuation: # No punctuation at all python -M numpy # Needs quotes - if quoted itself, needs nested quotes python -C "import numpy" # Needs quotes and semicolon python -c "import numpy; ..." This may not be a big deal on Unix shells (IMO, it's still an issue there, just less critical) but on less-capable shells like Windows' CMD, avoiding unnecessary punctuation can be a big improvement. Paul
Steve Barnes wrote:
It would be really cool if you could somehow write a file with a bunch of commands in it, and then get Python to execute those commands. Then it could still be a one line invocation, but you could do much more complex things, including import a bunch of modules before executing some code. I'm not sure what such a file would look like but here's a strawman: ``` import os import sys import somelib path = somelib.get_path() parent = os.path.dirname(path) print(parent) sys.exit(0 if os.path.isdir(parent) else 1) ``` Then you could run it like so: $ python3 myscript.py That seems like a nice, compact, one line invocation, but cI don't know, it probably needs some fleshing out. It's just a crazy idea, and there's probably not enough time to implement this for Python 3.7. Maybe for Python 3.8. time-machine-winking-ly y'rs, -Barry
On 10 January 2018 at 07:54, Barry Warsaw <barry@python.org> wrote:
You jest, but doing that and then going on to process the rest of the command line the same way the interpreter normally would is genuinely tricky. Even `python -m runpy [other args]` doesn't emulate it perfectly, and its responsible for implementing large chunks of the regular behaviour :) For the coverage.py use case, an environment-based solution is also genuinely helpful, since you typically can't modify subprocess invocations just because the software is being tested. At the moment, there are approaches that rely on using either `sitecustomize` or `*.pth` files, but being able to write `PYTHONRUNFIRST="import coverage; coverage.process_startup()"` would be a fair bit clearer about what was actually going on. That example also shows why I'm wary of offering an import-only version of this: I believe it would encourage folks to write modules that have side effects on import, which is something we try to avoid doing. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 10 January 2018 at 02:39, Nick Coghlan <ncoghlan@gmail.com> wrote:
It's worth remembering that Windows doesn't have the equivalent of the Unix "VAR=xxx prog arg arg" syntax for one-time setting of an environment variable, so environment variable based solutions are strictly less useful than command line arguments. That's one reason I prefer -C over PYTHONRUNFIRST. Paul
On 10 January 2018 at 18:30, Paul Moore <p.f.moore@gmail.com> wrote:
The proposal is to offer both, not an either/or, but the idea isn't that folks would need to set the environment variable directly - it's that coverage.py itself (or a test runner) would set it so that subprocesses were captured in addition to the directly executed module. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Could it just check if -c and -m are both set? That way there'd be no need for -p or -M. (I have an -m <module> switch in pyline which does exactly this. It makes copying and pasting less convenient; but does save having to type 'import module;’ for one liners) On Friday, January 5, 2018, Steve Barnes <gadgetsteve@live.co.uk> wrote:
On 5 January 2018 at 16:28, Steve Barnes <gadgetsteve@live.co.uk> wrote:
That's not quite how the -m switch works, but that doesn't affect your core point: "-m somemodule" terminates the option list and defines what will be run as `__main__`, so you can't compose it with other options (like `-c`).
This could be an interesting alternative to the old "python -C" idea in https://bugs.python.org/issue14803 (which suggests adding an alternative to "-c" that runs the given code *before* running `__main__`, rather than running it *as* `__main__`). To summarise some of the use cases from that discussion: * setting `__main__.__requires__` to change how `import pkg_resources` selects default package versions * reducing the proliferation of `-X` options to enable modules like tracemalloc and faulthandler * providing a way to enable code coverage as early as possible on startup (and have the setting be inherited by subprocesses) The main downside I'd see compared to that `python -C` and PYTHONRUNFIRST idea is that it might encourage the definition of modules with side-effects in order to use it to reconfigure the interpreter at startup. That concern could potentially be mitigated if we did both: * -M/PYTHONIMPORTFIRST: implicit imports in __main__ on startup * -C/PYTHONRUNFIRST: code to run in __main__ on startup (after the implicit imports) However, the issue then is that "python -M numpy" would just be a less flexible alternative to a command like "python -C 'import numpy as np'". Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 5 January 2018 at 08:12, Nick Coghlan <ncoghlan@gmail.com> wrote:
For quick one-liners don't underestimate the value of avoiding punctuation: # No punctuation at all python -M numpy # Needs quotes - if quoted itself, needs nested quotes python -C "import numpy" # Needs quotes and semicolon python -c "import numpy; ..." This may not be a big deal on Unix shells (IMO, it's still an issue there, just less critical) but on less-capable shells like Windows' CMD, avoiding unnecessary punctuation can be a big improvement. Paul
Steve Barnes wrote:
It would be really cool if you could somehow write a file with a bunch of commands in it, and then get Python to execute those commands. Then it could still be a one line invocation, but you could do much more complex things, including import a bunch of modules before executing some code. I'm not sure what such a file would look like but here's a strawman: ``` import os import sys import somelib path = somelib.get_path() parent = os.path.dirname(path) print(parent) sys.exit(0 if os.path.isdir(parent) else 1) ``` Then you could run it like so: $ python3 myscript.py That seems like a nice, compact, one line invocation, but cI don't know, it probably needs some fleshing out. It's just a crazy idea, and there's probably not enough time to implement this for Python 3.7. Maybe for Python 3.8. time-machine-winking-ly y'rs, -Barry
On 10 January 2018 at 07:54, Barry Warsaw <barry@python.org> wrote:
You jest, but doing that and then going on to process the rest of the command line the same way the interpreter normally would is genuinely tricky. Even `python -m runpy [other args]` doesn't emulate it perfectly, and its responsible for implementing large chunks of the regular behaviour :) For the coverage.py use case, an environment-based solution is also genuinely helpful, since you typically can't modify subprocess invocations just because the software is being tested. At the moment, there are approaches that rely on using either `sitecustomize` or `*.pth` files, but being able to write `PYTHONRUNFIRST="import coverage; coverage.process_startup()"` would be a fair bit clearer about what was actually going on. That example also shows why I'm wary of offering an import-only version of this: I believe it would encourage folks to write modules that have side effects on import, which is something we try to avoid doing. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 10 January 2018 at 02:39, Nick Coghlan <ncoghlan@gmail.com> wrote:
It's worth remembering that Windows doesn't have the equivalent of the Unix "VAR=xxx prog arg arg" syntax for one-time setting of an environment variable, so environment variable based solutions are strictly less useful than command line arguments. That's one reason I prefer -C over PYTHONRUNFIRST. Paul
On 10 January 2018 at 18:30, Paul Moore <p.f.moore@gmail.com> wrote:
The proposal is to offer both, not an either/or, but the idea isn't that folks would need to set the environment variable directly - it's that coverage.py itself (or a test runner) would set it so that subprocesses were captured in addition to the directly executed module. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (6)
-
Barry Warsaw -
Ivan Pozdeev -
Nick Coghlan -
Paul Moore -
Steve Barnes -
Wes Turner