Running a module as a script

Patch # 1035498 attempts to implement the semantics suggested by Ilya and Anthony and co. "python -m module" Means: - find the source file for the relevant module (using the standard locations for module import) - run the located script as __main__ (note that containing packages are NOT imported first - it's as if the relevant module was executed directly from the command line) - as with '-c' anything before the option is an argument to the interpreter, anything after is an argument to the script The allowed modules are those whose associated source file meet the normal rules for a command line script. I believe that means .py and .pyc files only (e.g. "python -m profile" works, but "python -m hotshot" does not). Special import hooks (such as zipimport) almost certainly won't work (since I don't believe they work with the current command line script mechanism). Cheers, Nick. -- Nick Coghlan Brisbane, Australia

On Thu, Sep 30, 2004, Nick Coghlan wrote:
Not positive, but if you're allowing .pyc, you should probably allow .pyo if optimize mode is on. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines." --Ralph Waldo Emerson

Quoting "Phillip J. Eby" <pje@telecommunity.com>:
Using the C equivalent of 'imp.find_module()' should cover all these cases, and any new forms of PY_SOURCE or PY_COMPILED that come up in future.
That's essentially what the patch does. PyRun_SimpleModuleFlags uses _PyImport_FindModule to obtain the absolute file location, then uses PyRun_SimpleFileExFlags to actually run the script. _PyImport_FindModule is a trivial wrapper around import.c's find_module() (which, I believe, is what imp.find_module() invokes) When multiple versions exist, the patch operates on what find_module() returns (which ends up using the preference order '.py', '.pyw', '.py[co]') Cheers, Nick. -- Nick Coghlan Brisbane, Australia

Quoting Nick Coghlan <ncoghlan@email.com>:
I've realised that this means that the '-m' option doesn't understand packages that modify __path__ in their __init__.py scripts. What do people think of semantics which say "python -m some.package.module" means that "some.package" gets imported before "module" gets executed as "__main__"? The advantages are that __path__ will be interpreted correctly and package initialisation code will be invoked before the module is executed. It seems slightly odd to be importing things before the script starts executing, but these semantics seem to be more in line with the behaviour of the rest of Python's import machinery. Cheers, Nick. -- Nick Coghlan Brisbane, Australia

At 02:47 PM 10/1/04 +1000, Nick Coghlan wrote:
Although it would be nice if the new functionality supported existing scripts, I'd almost rather see the semantics of '-m foo.bar' be the same as '-c "from foo.bar import __main__; __main__()"', since the latter's semantics are much more well-defined. Alternatively, one could use '-m foo.bar.__main__' or '-m timeit.main' or '-m pydoc.cli' or '-m unittest.main', and thus be explicit about exactly what will be run. In either case, though, I think import semantics are easier to explain/understand than __name__=='__main__' semantics, especially in situations where the "script" may be re-imported by other code it imports (e.g. the unittest module).

Quoting "Phillip J. Eby" <pje@telecommunity.com>:
Jim suggested something similar. The thing is, such idioms are already quite easy to handle using '-c' (see above for an example ;). What isn't possible is invoking the "if __name__ == '__main__':" idiom without knowing the location of a module in the file system. About the closest we get for Unix'y platforms is to run 'python `python -c "import inspect; import x; print inspect.getsourcefile(x)"`'. (if the quotes aren't clear, those are backticks around the python -c invocation). Anyway, I think -m really only makes sense if it gives us the power to invoke a module as "__main__". Otherwise we might as well stick to using -c. Cheers, Nick. -- Nick Coghlan Brisbane, Australia

On Thu, Sep 30, 2004, Nick Coghlan wrote:
Not positive, but if you're allowing .pyc, you should probably allow .pyo if optimize mode is on. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines." --Ralph Waldo Emerson

Quoting "Phillip J. Eby" <pje@telecommunity.com>:
Using the C equivalent of 'imp.find_module()' should cover all these cases, and any new forms of PY_SOURCE or PY_COMPILED that come up in future.
That's essentially what the patch does. PyRun_SimpleModuleFlags uses _PyImport_FindModule to obtain the absolute file location, then uses PyRun_SimpleFileExFlags to actually run the script. _PyImport_FindModule is a trivial wrapper around import.c's find_module() (which, I believe, is what imp.find_module() invokes) When multiple versions exist, the patch operates on what find_module() returns (which ends up using the preference order '.py', '.pyw', '.py[co]') Cheers, Nick. -- Nick Coghlan Brisbane, Australia

Quoting Nick Coghlan <ncoghlan@email.com>:
I've realised that this means that the '-m' option doesn't understand packages that modify __path__ in their __init__.py scripts. What do people think of semantics which say "python -m some.package.module" means that "some.package" gets imported before "module" gets executed as "__main__"? The advantages are that __path__ will be interpreted correctly and package initialisation code will be invoked before the module is executed. It seems slightly odd to be importing things before the script starts executing, but these semantics seem to be more in line with the behaviour of the rest of Python's import machinery. Cheers, Nick. -- Nick Coghlan Brisbane, Australia

At 02:47 PM 10/1/04 +1000, Nick Coghlan wrote:
Although it would be nice if the new functionality supported existing scripts, I'd almost rather see the semantics of '-m foo.bar' be the same as '-c "from foo.bar import __main__; __main__()"', since the latter's semantics are much more well-defined. Alternatively, one could use '-m foo.bar.__main__' or '-m timeit.main' or '-m pydoc.cli' or '-m unittest.main', and thus be explicit about exactly what will be run. In either case, though, I think import semantics are easier to explain/understand than __name__=='__main__' semantics, especially in situations where the "script" may be re-imported by other code it imports (e.g. the unittest module).

Quoting "Phillip J. Eby" <pje@telecommunity.com>:
Jim suggested something similar. The thing is, such idioms are already quite easy to handle using '-c' (see above for an example ;). What isn't possible is invoking the "if __name__ == '__main__':" idiom without knowing the location of a module in the file system. About the closest we get for Unix'y platforms is to run 'python `python -c "import inspect; import x; print inspect.getsourcefile(x)"`'. (if the quotes aren't clear, those are backticks around the python -c invocation). Anyway, I think -m really only makes sense if it gives us the power to invoke a module as "__main__". Otherwise we might as well stick to using -c. Cheers, Nick. -- Nick Coghlan Brisbane, Australia
participants (4)
-
Aahz
-
Nick Coghlan
-
Phillip J. Eby
-
Thomas Heller