[issue24632] Improve documentation about __main__.py
New submission from Ezio Melotti: __main__.py seems to only be mentioned briefly in a couple of places in the official docs: 1) https://docs.python.org/3/library/__main__.html 2) https://docs.python.org/3/using/cmdline.html#cmdoption-m The first link only says: """For a package, the same effect can be achieved by including a __main__.py module, the contents of which will be executed when the module is run with -m.""" ("-m" should actually use :option:`-m` to automatically link to the second URL.) The second link mentions __main__.py in two sentences: """Execute the Python code contained in script, which must be a filesystem path (absolute or relative) referring to either a Python file, a directory containing a __main__.py file, or a zipfile containing a __main__.py file.""" """If the script name refers to a directory or zipfile, the script name is added to the start of sys.path and the __main__.py file in that location is executed as the __main__ module.""" I think it would be better to expand the first link to state clearly what is __main__.py and what is its purpose. In addition, the section should clarify a few more things, e.g. when it should be used, what it should contain, if it's ok to have other __main__.py in the subpackages (e.g. test/__main__.py to run the tests with python -m package.test), how it interacts __init__.py (which one is executed first?). Perhaps it should also get a glossary entry and/or a short mention in the tutorial together with zip imports. In addition to the two links above, a Google search returns the stackoverflow question "What is __main__.py?" as first result, and a couple more related questions that could/should be answered by our docs. ---------- assignee: docs@python components: Documentation messages: 246719 nosy: docs@python, ethan.furman, ezio.melotti, nedbat priority: normal severity: normal stage: needs patch status: open title: Improve documentation about __main__.py type: enhancement versions: Python 2.7, Python 3.4, Python 3.5 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue24632> _______________________________________
Ned Batchelder added the comment: BTW, the Stack Overflow answer: http://stackoverflow.com/a/4043007 ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue24632> _______________________________________
Ezio Melotti added the comment: Another thing that should be clarified, is the difference between __main__.py inside a package, and __main__.py inside a zip file. For packages, as far as I understand, __main__.py should be inside the package (i.e. pkg/__main__.py, in the same dir of pkg/__init__.py). This allows the package to be "executed" by doing "python3 -m pkg". For zip files, the __main__.py should be right inside the zip (i.e. file.zip/__main__.py). This allows the zip file to be "executed" by doing "python3 file.zip" (note that no -m is used here, and that "python3 -m file.zip" fails). While zipping a package that already contains a __main__.py, the right way to do it seems to be the following: 1) add the package to a zip file (i.e. file.zip/pkg/) 2) add another __main__.py to the zip (i.e. file.zip/__main__.py) 3) add 'import pkg.__main__' to file.zip/__main__.py now if you do "python3 file.zip", file.zip/__main__.py will be executed, and in turn it will import file.zip/pkg/__main__.py, obtaining a result equivalent to "python -m pkg". (I still haven't figured out if the __main__.py is necessary while /importing/ a package/module from a zip file, after having added the zip file to sys.path.) So, to summarize, it seems to me that: 1) pkg/__main__.py is necessary to run "python3 -m pkg" (with -m); 2) file.zip/__main__.py is necessary to run "python3 file.zip" (without -m); 3) due to 1) and 2) creating an executable zipped package requires 2 __main__.py; 4) "python3 pkg" and "python3 -m file.zip" are not supposed to work even if the __main__.py files are in the right place. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue24632> _______________________________________
Ezio Melotti added the comment: After further tests, I think I figured out how things works. There are three separate things that interact with each other: * packages (dirs with an __init__.py) and "regular" dirs (with no __init__.py) or zip files; * how python is executed (with or without -m); * if the pkg/dir/zip is executed or imported. __main__.py makes a pkg/dir/zip "executable", but: * if it's a package, "python -m pkg" should be used; * if it's a dir or zip, "python dir_or_zip" should be used instead. There seem to be no differences between "regular" dirs and zip files: * both can become executable with a __main__.py; * both should be executed with "python dir_or_zip" (no -m); * both can not be imported (if we ignore namespace packages); * both can be added to sys.path, and the modules they contain imported, without needing any __main__.py. This also means that __main__.py is used only while doing "python -m pkg" or "python dir_or_zip", and not while doing "import pkg" or while importing a module inside a dir/zip. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue24632> _______________________________________
R. David Murray added the comment: What you just described is exactly what I would have said was the case (a zip file acts exactly like it was a directory), so I'm glad that's the way it actually works :). ---------- nosy: +r.david.murray _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue24632> _______________________________________
R. David Murray added the comment: The surprising thing is that __main__ works without there being an __init__. I didn't know that, assumed it wasn't true, and so never tried it. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue24632> _______________________________________
Ezio Melotti added the comment:
The surprising thing is that __main__ works without there being an __init__.
That's also what surprised me, I always thought __main__.py was supposed to be used within a package executed with "python -m pkg", but apparently "regular" dirs and zip files can have one too -- as long as they are executed as "python dir_or_zip". This should have answered the question I posed in my first message: what is __main__.py and what is its purpose? As for the others: Q: when should it be used? A: whenever you want to make a package/dir/zip executable Q: what should it contain? A: usually an import + a function call that launches the app should be enough, but might contain more code if necessary Q: is it ok to have other __main__.py in the subpackages (e.g. test/__main__.py to run the tests with python -m package.test)? A: this seems to work and should be OK Q: how it interacts __init__.py (which one is executed first?) A: __init__.py seems to be executed first. I'm not aware of other interactions. If these are indeed correct, a patch can be made (feel free to do it, since I don't when I'll have time to do it myself). ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue24632> _______________________________________
Ethan Furman added the comment: RDM noted: ---------
The surprising thing is that __main__ works without there being an __init__. I didn't know that, assumed it wasn't true, and so never tried it.
I think this is due to PEP 420 Namespace Packages. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue24632> _______________________________________
Ezio Melotti added the comment:
I think this is due to PEP 420 Namespace Packages.
It works on Python 2 too: $ ls execdir/ foo.py __main__.py $ cat execdir/foo.py print("foo imported") $ cat execdir/__main__.py import foo; print("main imported") $ python execdir/ foo imported main imported $ python -V Python 2.7.8 I haven't done any tests about the interaction of namespace packages and __main__.py, but if there are additional semantics, they should be documented as well. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue24632> _______________________________________
Davide Rizzo added the comment: As far as I understand, assuming dir/ contains a __main__.py file $ python dir is equivalent to $ python dir/__main__.py in that it's behaviourally nothing more than executing a script in that dir and setting sys.path accordingly. This is the same in Python 2 and Python 3. This, together with the notion that zip files and directories are treated in the same way, allows running python file.zip since we have no option for executing a file *within* the zip file. Altogether, this is a significantly different behaviour than the one for "python -m pkg". That would be closer to:
import pkg.__main__
This also explains why the package __init__ is executed first (you import the package first, then the module). A significant difference is that it's not a real import (just as pkg.__init__ is not imported) and sys.modules is not affected. ---------- nosy: +davide.rizzo _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue24632> _______________________________________
Changes by Ethan Furman <ethan@stoneleaf.us>: ---------- nosy: -ethan.furman _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue24632> _______________________________________
Changes by Ezio Melotti <ezio.melotti@gmail.com>: ---------- nosy: +ncoghlan versions: +Python 3.6 -Python 3.4 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue24632> _______________________________________
Dima Tisnek added the comment: +1, I too would like to see this documented ---------- nosy: +Dima.Tisnek _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue24632> _______________________________________
Changes by Mariatta Wijaya <mariatta.wijaya@gmail.com>: ---------- nosy: +Mariatta _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue24632> _______________________________________
Nick Coghlan added the comment: Yeah, I never found a good place to document this, hence the relatively sparse references in the "using" docs. The most complete official docs for these features are actually in runpy: * https://docs.python.org/3/library/runpy.html#runpy.run_module * https://docs.python.org/3/library/runpy.html#runpy.run_path run_module is a thin wrapper around the same core code that actually implements the -m switch run_path is a Python level reimplementation of the __main__ execution logic There's also http://www.curiousefficiency.org/posts/2011/03/what-is-python-script.html on my blog and the explanation of the options that need to be supported in PEP 432: https://www.python.org/dev/peps/pep-0432/#preparing-the-main-module A lot of the confusion stems from the fact that directory & zipfile execution was added without a PEP back in 2.6 (it was just a normal tracker issue) and we forgot to add it to the What's New document. We hoped the inclusion of the zipapp module in Python 3.5 via PEP 441 might help resolve that lack of awareness, but it doesn't seem to have had much impact. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue24632> _______________________________________
Nick Coghlan added the comment: In the same vein of "I never worked out a good offical home for it", a couple of the "Traps for the Unwary" I describe for Python's import system are closely related to __main__ module execution and the impact that has on sys.path and sys.modules: * http://python-notes.curiousefficiency.org/en/latest/python_concepts/import_t... * http://python-notes.curiousefficiency.org/en/latest/python_concepts/import_t... ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue24632> _______________________________________
Mitch Gann <mgann85@gmail.com> added the comment: Can someone please update the Docs on this? Its been over 3 years since the issue was raised and seems fairly straightforward to address. Also considering its a fairly important feature of the language which deals with the entry point for packages/modules to be executed, I know many Engineers/Developers using and learning the language would benefit from official documentation on this aspect. ---------- nosy: +MGann85 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue24632> _______________________________________
participants (9)
-
Davide Rizzo -
Dima Tisnek
-
Ethan Furman
-
Ezio Melotti
-
Mariatta Wijaya
-
Mitch Gann
-
Ned Batchelder
-
Nick Coghlan
-
R. David Murray