'-m' option (was RE: [Python-Dev] ConfigParser patches)

[Nick Coghlan]

Aside for Carlos: 'm' is short for 'module', and beefing up Python's option
parsing to handle things like '--module' is a whole 'nother story. 'python -h'
gives a decent description of all the options, though.

Guido van Rossum wrote:
>>Pros (-m over runpy.py, in order of significance as I see it):
>>  - easy to combine with other Python command line options
> 
> The script approach can do this too

Only by spelling out the invocation if we want something non-standard.

Using the laptop I'm typing on as an example. . .

Python 2.3 is the installed version, so "#!/usr/bin/env python" in a script will
invoke Py 2.3 without any extra options

If I (or a Python installer) were to create "/usr/local/bin/runpy.py" with that
shebang line (i.e. the same as the one you posted), we have the following for
profiling a sample script in my current directory with different versions of the
interpreter:

Installed:     runpy.py profile demo.py
Prompt After:  python -i /usr/local/bin/runpy.py profile demo.py
Alt install:   python2.4 /usr/local/bin/runpy.py profile demo.py
Build dir:     ./python /usr/local/bin/runpy.py profile demo.py

If we wanted to use the version of the script that came with the relevant
version of python, those last two would become:

Alt install*:
  python2.4 /usr/local/lib/python2.4/runpy.py profile demo.py
Build dir:     ./python Tools/scripts/runpy.py profile demo.py

(* This is based on what happens to pydoc under 'make altinstall'. The shebang
line is version agnostic, so it tries to use the Py2.3 interpreter with the
Py2.4 library modules and it all falls apart. So to run a library module of the
altinstall, this is how I have to do it. And this is assuming the script would
get installed at all, which it may not, as it isn't actually a library module,
unlike pydoc)


Using -m, those become:

Installed:     python -m profile demo.py
Prompt After:  python -i -m profile demo.py
Alt install:   python24 -m profile demo.py
Build dir:     ./python -m profile demo.py

>>  - OS & environment independent
> 
> So is the script -- probably more so, since the script can use
> Python's OS independence layer.

Paul's message goes into detail on what I meant here. The script itself is
highly portable, the mechanism for invoking it really isn't.

The C code in the patch is platform independent - _PyImport_FindModule (a
trivial wrapper around the existing find_module in import.c) and
PyRun_SimpleFileExFlags do all the heavy lifting. In fact, sans error-checking,
the guts of PyRun_SimpleModuleFlags looks remarkably similar to the Python code
in your script.

(I initially thought the Python version might handle zip imports, while the C
version didn't. However, a quick experiment shows that *neither* of them can
handle zip imports. And the source code confirms it - imp.find_module and
imp.load_module don't allow zip imports, and PyRun_Module in the patch doesn't
allow them either. Amusingly, the current limitations of the imp module make it
easier to support zip imports with the *C* version. Allowing imp.find_module to
return a 4th value for the module loader would require adding an optional
boolean argument to avoid breaking existing code, whereas the patch's new
private C API for _PyImport_FindModule already exposes the loader argument)

>>  - more concise to invoke
> Depends on the length of the name of the script. :-)

See the examples above for what I meant with this one. For the vanilla case
you're right, but as soon as we do anything slightly different, the story changes.

>>  - no namespace issues with naming a script
> Actually, one-letter options are a much scarcer resource than script names.

Well, with '-m' in place, we'd be using 17 out of the available 62 (upper &
lower alpha, plus digits). The difference is that we're only competing with
ourselves and the other Python interpreter authors for characters to use,
whereas we're competing with all and sundry for unique executable names.
(Windows isn't immune, either, given enough applications with directories on
PATH. Although retaining the '.py' helps a lot there)

(I have checked that Jython at least doesn't use '-m' for anything. I don't know
about other interpreters)

>>  - C API for those embedding python
> And who needs that?

There's a reason this one was last on my list :)

> Additional Pros for using a script:
>  - less code to maintain

Once we factor in the additional packaging requirements to make the script as
easy to use on all target platforms as -m would be, I think this one is at least
arguable (script + packaging vs option-parsing and C function).

>  - can work with older Python versions
>  - shows users how to do a similar thing themselves with additional
> features

Certainly, dropping a version of this script into Tools/scripts in CVS couldn't
hurt, regardless of whether or not '-m' gets adopted. The same would go for an
entry in the Python cookbook.

> (e.g. a common addition I expect will be to hardcode a
> personal sys.path addition)

Except that this feature isn't so much for your *own* scripts, as it is for
installed modules that are also usable as scripts (like profile and pdb). In the
former case, you *know* where those scripts are (for me ~/script_name usually
does the trick on *nix). In the latter case, though, it'd be nice to be able to
use these things easily 'out of the box'.

For those who want to tweak the search behaviour, all the usual environment
variables apply (PYTHONPATH in particular).

Heck, there's nothing to stop someone from doing something like the following if
they really want to:

python -m my_run_module_script some_other_module

The command line interface is one of the major holdouts in Python where we
really need to care where the source file for a module lives. It'd be nice to
change that.

Cheers,
Nick.

-- 
Nick Coghlan
Brisbane, Australia