Global flag for whether a module is __main__
Currently, the simplest and most idiomatic way to check whether a module was run as a script rather than imported is: if __name__ == "__main__": People generally learn this by rote memorization, because users often want the ability to add testing code or command line interfaces to their modules before they understand enough about Python's data model to have any idea why this works. Understanding what's actually happening requires you to know that: 1. the script you ask Python to run is technically a module, 2. every module has a unique name assigned to it, 3. a module's `__name__` global stores this unique import name, 4. and "__main__" is a magic name for the initial script's module. A new (writable) global attribute called `__main__` would simplify this case, allowing users to simply test if __main__: It would behave as though __main__ = (__name__ == "__main__") is executed in each module's namespace before executing it. Because this would be writable, I don't see any backwards compatibility issues. It wouldn't negatively impact any modules that might already be defining `__main__` (for example, by doing `import __main__`). They'd simply redefine it and go on using the `__main__` module as they always have. And a package with a `__main__.py` does not have a `__main__` attribute. It would be easier to teach, easier to learn, and easier to memorize, and a nice simplification for users at the cost of only very slightly more complexity in the data model.
On Fri, Nov 13, 2020 at 12:47 AM Matt Wozniski <godlygeek@gmail.com> wrote:
Benefits: One incantation becomes slightly shorter. Costs: Forever have to maintain both ways of doing things. The __name__ version of the idiom is going to stick around (there's no point breaking people's code), so there's going to be two ways to spell the exact same thing. Both are still going to have to be memorized (you have to get exactly two underscores either side), and while the alternate IS shorter, it's not that much of a benefit. -0.9. I actually don't use the "if name is main" idiom all that often. The need to have a script be both a module and an executable is less important than you might think. In a huge number of cases, it's actually better to separate out the library-like and script-like portions into separate files, or some other reorganization. It just isn't that much benefit, especially since the longhand will be essential for compatibility with all versions up to X.Y when the shorthand would get added. ChrisA
On 11/12/20, Chris Angelico <rosuav@gmail.com> wrote:
One caveat is that the module __name__ check is required for multiprocessing in spawn mode (as opposed to fork mode), which is the only supported mode in Windows. Generally, I think scripts in installed packages are better handled via setuptools entrypoints nowadays. For cross-platform support, pip automatically does the right thing for Windows by creating executable .exe launchers. The issue is two-fold: the default action of the .py file association is often configured to edit rather than execute scripts, and, irrespective of the latter, many execution paths, such as subprocess.Popen, use CreateProcess from the base API, which does not support file associations. (File associations are the closest Windows has to Unix shebangs, and the basis for how the py.exe launcher supports Unix shebangs in scripts, but they're implemented in the high-level shell API instead of the base API.)
I kinda like it. Definitely +0.0, maybe +0.5. On Thu, Nov 12, 2020 at 2:46 PM Matt Wozniski <godlygeek@gmail.com> wrote:
Some other commenters say they rarely dual-use modules as scripts. I do this quite often myself. There's some little functionality that is useful from the command-line, but some supporting functions within it, or some more nuanced version of calling those arguments is also useful in larger projects. I know there are other ways to achieve that purpose, but for quick-and-easy, the __name__=='__main__' is something I do often. I don't think this would break anything. Yes, the slightly more verbose version would need to be maintained forever. But the semantics of __main__ being implicitly defined when scripts are run is easy to explain. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
David Mertz writes:
[T]he semantics of __main__ being implicitly defined when scripts are run is easy to explain.
Especially to C programmers on Unix. ;-) To be honest, I think this is pretty icky[1]. IMO, it would be more consistent with the Python runtime if it were a function is_main() and lived in sys. The existing idiom is a sort of easter egg, but it is (almost[2]) implied by the structure and semantics of modules, and almost as easy to explain as "if __main__" would be. __main__ *appears* to be easy to explain because the syntax is so simple, but it's not obvious (to me, anyway) what it should be inside of eval(). And is it an attribute of the script itself (which new users don't necessarily think of as an object in Python), or of builtins, which is where most builtin identifiers live? Pretty clearly it's part of the module namespace, but to many new users that's going to require an explanation almost as deep as "if __name__ == '__main__'" does.
I don't think this would break anything.
Just my heart! ;-) Footnotes: [1] I've been lifetime banned from use of the adjective "un-Pythonic". :-) [2] You need the small additional assumption that __name__ is initialized to a specific value when running outside of the import system.
Hello, On Thu, 12 Nov 2020 07:56:17 -0000 "Matt Wozniski" <godlygeek@gmail.com> wrote:
But that's absolutely great! A novice can first memorize the 'if __name__ == "__main__":' idiom, but it will plant seed for them to explore Python module and module loading systems. Such ways of learning is what makes Python great.
If anything, that conflicts with other proposed usages for __main__, e.g. https://mail.python.org/archives/list/python-ideas@python.org/thread/FBT5BT7... Having a function __main__() is definitely more useful than an alias for trivial comparison op. [] -- Best regards, Paul mailto:pmiscml@gmail.com
Hello, On Thu, 12 Nov 2020 18:33:52 +0000 Rob Cliffe via Python-ideas <python-ideas@python.org> wrote: []
Where have you seen 10kg? "Here're your mocassins, they're lightweight and nice, though may be not what you wore in your 'hood. You can just wear them now and experience comfort they offer, but can also use them as a hint to learn more about the cultures of the peoples of the Earth." Goes offtopic tho. [] -- Best regards, Paul mailto:pmiscml@gmail.com
On Thu, Nov 12, 2020 at 06:33:52PM +0000, Rob Cliffe via Python-ideas wrote:
Do you honestly think that writing if __name__ == '__main__' is equivalent to a 10kg pair of shoes? If that is the case, then the suggestion to use `if __main__` alone is surely 7kg, and writing a function with `def` is probably 2000 kg. Fortunately Python coders are mighty, with powerful muscles, who think nothing of lifting thousands of tonnes at a time. All joking, and bad analogies, aside, I am disturbed at the idea that expecting Python programmers to understand what the code they wrote actually does, and what the code means, is a bad thing. Sure, there are plenty of people who, due to time constraints, lack of interest, or some other reason, *don't care* why the `if __name__` idiom works or what it does. For them, they can just memorize it as a magic incantation, or copy it from Stackoverflow as required. I have plenty of code like that myself. But I'm not convinced that we should add a language feature specifically to enable that behaviour, just to save a few characters of typing. For years, as a beginner, I used to forget the string quotes: if __name__ == __main__: Oops! But at least I got a NameError and my mistake was then obvious. With this suggested feature, the code will just silently fail. -- Steve
Also see PEP 299 for a slightly different approach.
Indeed. This proposal is more limited in scope than PEP 299, and appears better from a backwards compatibility perspective.
there's going to be two ways to spell the exact same thing. Both are still going to have to be memorized
Sure - though ideally over time the more verbose spelling will become less idiomatic, and new users will encounter it less, and learners won't need to memorize it because they won't need it and won't see it. Granted there would be several years between the new feature being introduced and when it is supported ubiquitously enough that the more verbose spelling becomes rare.
Other than the note about multiprocessing's spawn mode, I personally agree with this. But still: this is extremely heavily used. There are almost 32 million hits for this idiom on GitHub: https://github.com/search?q=%22__name__+%3D%3D+__main__&type=code It seems that many people find value in this, and so lots of new users continue to be exposed to it.
Generally, I think scripts in installed packages are better handled via setuptools entrypoints nowadays.
In my experience, people reach the point of splitting their own scripts out into private modules long before they reach the point of packaging things. I propose this because I think it makes a gentler path for learners. Pushing them in the direction of setuptools instead gives them a brand new hurdle to overcome, one that's even more complex than the rote memorization I'd like to see overcome.
On Thu, Nov 12, 2020 at 9:46 AM Matt Wozniski <godlygeek@gmail.com> wrote:
As a goal of making it even more obvious what the (new) idiom mans, I would suggest a variable named __imported__ with the opposite value to what is proposed. Thus, if not __imported__: would behave the same as the proposed if __main__: André
Excellent redirect of original proposal: +10 From: André Roberge <andre.roberge@gmail.com> [snip] As a goal of making it even more obvious what the (new) idiom mans, I would suggest a variable named __imported__ with the opposite value to what is proposed. Thus, if not __imported__: would behave the same as the proposed if __main__: André
I love the idea of __imported__ but would like to suggest taking it even further - why not, rather that a bool for __imported__, have the value of it be either None (if __name__ == "__main__") or the __name__ of the module that it is being imported from. This could potentially be useful in trying to resolve circular imports, imports via importlib or exec, etc. The code need would be slightly more complex needing to check the current call stack at the time of import but I don't think that it would be too much overhead. Steve Barnes From: Henshaw, Andy via Python-ideas <python-ideas@python.org> Sent: 13 November 2020 16:06 To: André Roberge <andre.roberge@gmail.com>; Matt Wozniski <godlygeek@gmail.com> Cc: Python-Ideas <python-ideas@python.org> Subject: [Python-ideas] Re: Global flag for whether a module is __main__ Excellent redirect of original proposal: +10 From: André Roberge <andre.roberge@gmail.com<mailto:andre.roberge@gmail.com>> [snip] As a goal of making it even more obvious what the (new) idiom mans, I would suggest a variable named __imported__ with the opposite value to what is proposed. Thus, if not __imported__: would behave the same as the proposed if __main__: André
On Sat, Nov 14, 2020 at 04:32:20AM +0000, Steve Barnes wrote:
What if you imported `__main__`?
or the __name__ of the module that it is being imported from.
What if it has been imported from many modules, as most modules will be?
"Potentially" useful? How? By the way, exec doesn't do an import of the code being run. -- Steve
On Fri, Nov 13, 2020 at 05:59:29AM -0400, André Roberge wrote:
What if you import the `__main__` module? What does `__imported__` say now, and how do you check for "running as a script" if `__main__` has imported itself -- or some other module has imported it? Is this `__imported__` variable *instead of* or *as well as* the proposed `__main__`? Below, I'm going to assume this variable is addition to the proposed `__main__`. If that's not the case, and instead this is just a change of spelling, my comments below are probably irrelevant.
So now we have three idioms for precisely the same thing. That's not merely three alternative calculation methods, like we might have here: result = x**2 + 5*x result = x*x + 5*x result = x*(x + 5) but three special purpose idioms which were intentionally created to mean the same thing. I hope you have a StackOverflow account so you will be around to answer the many questions asking what the difference between them are. I don't know about you, but I never remember what are the differences between `type(obj)` and `obj.__class__`. So I'm sure people will have the same problem here. Might I suggest this? "The `__name__ = ...` idiom is the legacy method. It will never go away because of backwards compatibility, and also because it is useful for a module to check its own name for other purposes. "The `__main__` variable is there as a shortcut. "And the `__imported__` variable is there because the Python core developers think that there is a large number of Python users who can't work out how to use `not` to flip a boolean flag." *wink* -- Steve
On Sat, Nov 14, 2020 at 7:56 AM Steven D'Aprano <steve@pearwood.info> wrote:
Running a module (no matter what its name is) from a command line would set __imported__ to False for that module. Using import some_module (or __import__("some_module")) would set some_module.__imported__ to True.
For beginners, I believe that it would be easier to understand what it means, which is the reason I had used something very similar in a project that has been shelved for the foreseeable future, https://aroberge.github.io/avantpy/docs/html/notimported.html André
On Sat, Nov 14, 2020 at 08:10:44AM -0400, André Roberge wrote:
Do you understand that a module can be both run and imported at the same time? # example.py import __main__ print(__main__.__file__) If you save that snippet as "example.py", and then run it: python3 example.py you have an example of a module that is being run and imported simultaneously. -- Steve
On Fri, Nov 13, 2020 at 12:47 AM Matt Wozniski <godlygeek@gmail.com> wrote:
Benefits: One incantation becomes slightly shorter. Costs: Forever have to maintain both ways of doing things. The __name__ version of the idiom is going to stick around (there's no point breaking people's code), so there's going to be two ways to spell the exact same thing. Both are still going to have to be memorized (you have to get exactly two underscores either side), and while the alternate IS shorter, it's not that much of a benefit. -0.9. I actually don't use the "if name is main" idiom all that often. The need to have a script be both a module and an executable is less important than you might think. In a huge number of cases, it's actually better to separate out the library-like and script-like portions into separate files, or some other reorganization. It just isn't that much benefit, especially since the longhand will be essential for compatibility with all versions up to X.Y when the shorthand would get added. ChrisA
On 11/12/20, Chris Angelico <rosuav@gmail.com> wrote:
One caveat is that the module __name__ check is required for multiprocessing in spawn mode (as opposed to fork mode), which is the only supported mode in Windows. Generally, I think scripts in installed packages are better handled via setuptools entrypoints nowadays. For cross-platform support, pip automatically does the right thing for Windows by creating executable .exe launchers. The issue is two-fold: the default action of the .py file association is often configured to edit rather than execute scripts, and, irrespective of the latter, many execution paths, such as subprocess.Popen, use CreateProcess from the base API, which does not support file associations. (File associations are the closest Windows has to Unix shebangs, and the basis for how the py.exe launcher supports Unix shebangs in scripts, but they're implemented in the high-level shell API instead of the base API.)
I kinda like it. Definitely +0.0, maybe +0.5. On Thu, Nov 12, 2020 at 2:46 PM Matt Wozniski <godlygeek@gmail.com> wrote:
Some other commenters say they rarely dual-use modules as scripts. I do this quite often myself. There's some little functionality that is useful from the command-line, but some supporting functions within it, or some more nuanced version of calling those arguments is also useful in larger projects. I know there are other ways to achieve that purpose, but for quick-and-easy, the __name__=='__main__' is something I do often. I don't think this would break anything. Yes, the slightly more verbose version would need to be maintained forever. But the semantics of __main__ being implicitly defined when scripts are run is easy to explain. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
David Mertz writes:
[T]he semantics of __main__ being implicitly defined when scripts are run is easy to explain.
Especially to C programmers on Unix. ;-) To be honest, I think this is pretty icky[1]. IMO, it would be more consistent with the Python runtime if it were a function is_main() and lived in sys. The existing idiom is a sort of easter egg, but it is (almost[2]) implied by the structure and semantics of modules, and almost as easy to explain as "if __main__" would be. __main__ *appears* to be easy to explain because the syntax is so simple, but it's not obvious (to me, anyway) what it should be inside of eval(). And is it an attribute of the script itself (which new users don't necessarily think of as an object in Python), or of builtins, which is where most builtin identifiers live? Pretty clearly it's part of the module namespace, but to many new users that's going to require an explanation almost as deep as "if __name__ == '__main__'" does.
I don't think this would break anything.
Just my heart! ;-) Footnotes: [1] I've been lifetime banned from use of the adjective "un-Pythonic". :-) [2] You need the small additional assumption that __name__ is initialized to a specific value when running outside of the import system.
Hello, On Thu, 12 Nov 2020 07:56:17 -0000 "Matt Wozniski" <godlygeek@gmail.com> wrote:
But that's absolutely great! A novice can first memorize the 'if __name__ == "__main__":' idiom, but it will plant seed for them to explore Python module and module loading systems. Such ways of learning is what makes Python great.
If anything, that conflicts with other proposed usages for __main__, e.g. https://mail.python.org/archives/list/python-ideas@python.org/thread/FBT5BT7... Having a function __main__() is definitely more useful than an alias for trivial comparison op. [] -- Best regards, Paul mailto:pmiscml@gmail.com
Hello, On Thu, 12 Nov 2020 18:33:52 +0000 Rob Cliffe via Python-ideas <python-ideas@python.org> wrote: []
Where have you seen 10kg? "Here're your mocassins, they're lightweight and nice, though may be not what you wore in your 'hood. You can just wear them now and experience comfort they offer, but can also use them as a hint to learn more about the cultures of the peoples of the Earth." Goes offtopic tho. [] -- Best regards, Paul mailto:pmiscml@gmail.com
On Thu, Nov 12, 2020 at 06:33:52PM +0000, Rob Cliffe via Python-ideas wrote:
Do you honestly think that writing if __name__ == '__main__' is equivalent to a 10kg pair of shoes? If that is the case, then the suggestion to use `if __main__` alone is surely 7kg, and writing a function with `def` is probably 2000 kg. Fortunately Python coders are mighty, with powerful muscles, who think nothing of lifting thousands of tonnes at a time. All joking, and bad analogies, aside, I am disturbed at the idea that expecting Python programmers to understand what the code they wrote actually does, and what the code means, is a bad thing. Sure, there are plenty of people who, due to time constraints, lack of interest, or some other reason, *don't care* why the `if __name__` idiom works or what it does. For them, they can just memorize it as a magic incantation, or copy it from Stackoverflow as required. I have plenty of code like that myself. But I'm not convinced that we should add a language feature specifically to enable that behaviour, just to save a few characters of typing. For years, as a beginner, I used to forget the string quotes: if __name__ == __main__: Oops! But at least I got a NameError and my mistake was then obvious. With this suggested feature, the code will just silently fail. -- Steve
Also see PEP 299 for a slightly different approach.
Indeed. This proposal is more limited in scope than PEP 299, and appears better from a backwards compatibility perspective.
there's going to be two ways to spell the exact same thing. Both are still going to have to be memorized
Sure - though ideally over time the more verbose spelling will become less idiomatic, and new users will encounter it less, and learners won't need to memorize it because they won't need it and won't see it. Granted there would be several years between the new feature being introduced and when it is supported ubiquitously enough that the more verbose spelling becomes rare.
Other than the note about multiprocessing's spawn mode, I personally agree with this. But still: this is extremely heavily used. There are almost 32 million hits for this idiom on GitHub: https://github.com/search?q=%22__name__+%3D%3D+__main__&type=code It seems that many people find value in this, and so lots of new users continue to be exposed to it.
Generally, I think scripts in installed packages are better handled via setuptools entrypoints nowadays.
In my experience, people reach the point of splitting their own scripts out into private modules long before they reach the point of packaging things. I propose this because I think it makes a gentler path for learners. Pushing them in the direction of setuptools instead gives them a brand new hurdle to overcome, one that's even more complex than the rote memorization I'd like to see overcome.
On Thu, Nov 12, 2020 at 9:46 AM Matt Wozniski <godlygeek@gmail.com> wrote:
As a goal of making it even more obvious what the (new) idiom mans, I would suggest a variable named __imported__ with the opposite value to what is proposed. Thus, if not __imported__: would behave the same as the proposed if __main__: André
Excellent redirect of original proposal: +10 From: André Roberge <andre.roberge@gmail.com> [snip] As a goal of making it even more obvious what the (new) idiom mans, I would suggest a variable named __imported__ with the opposite value to what is proposed. Thus, if not __imported__: would behave the same as the proposed if __main__: André
I love the idea of __imported__ but would like to suggest taking it even further - why not, rather that a bool for __imported__, have the value of it be either None (if __name__ == "__main__") or the __name__ of the module that it is being imported from. This could potentially be useful in trying to resolve circular imports, imports via importlib or exec, etc. The code need would be slightly more complex needing to check the current call stack at the time of import but I don't think that it would be too much overhead. Steve Barnes From: Henshaw, Andy via Python-ideas <python-ideas@python.org> Sent: 13 November 2020 16:06 To: André Roberge <andre.roberge@gmail.com>; Matt Wozniski <godlygeek@gmail.com> Cc: Python-Ideas <python-ideas@python.org> Subject: [Python-ideas] Re: Global flag for whether a module is __main__ Excellent redirect of original proposal: +10 From: André Roberge <andre.roberge@gmail.com<mailto:andre.roberge@gmail.com>> [snip] As a goal of making it even more obvious what the (new) idiom mans, I would suggest a variable named __imported__ with the opposite value to what is proposed. Thus, if not __imported__: would behave the same as the proposed if __main__: André
On Sat, Nov 14, 2020 at 04:32:20AM +0000, Steve Barnes wrote:
What if you imported `__main__`?
or the __name__ of the module that it is being imported from.
What if it has been imported from many modules, as most modules will be?
"Potentially" useful? How? By the way, exec doesn't do an import of the code being run. -- Steve
On Fri, Nov 13, 2020 at 05:59:29AM -0400, André Roberge wrote:
What if you import the `__main__` module? What does `__imported__` say now, and how do you check for "running as a script" if `__main__` has imported itself -- or some other module has imported it? Is this `__imported__` variable *instead of* or *as well as* the proposed `__main__`? Below, I'm going to assume this variable is addition to the proposed `__main__`. If that's not the case, and instead this is just a change of spelling, my comments below are probably irrelevant.
So now we have three idioms for precisely the same thing. That's not merely three alternative calculation methods, like we might have here: result = x**2 + 5*x result = x*x + 5*x result = x*(x + 5) but three special purpose idioms which were intentionally created to mean the same thing. I hope you have a StackOverflow account so you will be around to answer the many questions asking what the difference between them are. I don't know about you, but I never remember what are the differences between `type(obj)` and `obj.__class__`. So I'm sure people will have the same problem here. Might I suggest this? "The `__name__ = ...` idiom is the legacy method. It will never go away because of backwards compatibility, and also because it is useful for a module to check its own name for other purposes. "The `__main__` variable is there as a shortcut. "And the `__imported__` variable is there because the Python core developers think that there is a large number of Python users who can't work out how to use `not` to flip a boolean flag." *wink* -- Steve
On Sat, Nov 14, 2020 at 7:56 AM Steven D'Aprano <steve@pearwood.info> wrote:
Running a module (no matter what its name is) from a command line would set __imported__ to False for that module. Using import some_module (or __import__("some_module")) would set some_module.__imported__ to True.
For beginners, I believe that it would be easier to understand what it means, which is the reason I had used something very similar in a project that has been shelved for the foreseeable future, https://aroberge.github.io/avantpy/docs/html/notimported.html André
On Sat, Nov 14, 2020 at 08:10:44AM -0400, André Roberge wrote:
Do you understand that a module can be both run and imported at the same time? # example.py import __main__ print(__main__.__file__) If you save that snippet as "example.py", and then run it: python3 example.py you have an example of a module that is being run and imported simultaneously. -- Steve
participants (12)
-
André Roberge -
Chris Angelico -
David Mertz -
Eric V. Smith -
Eryk Sun -
Henshaw, Andy -
Matt Wozniski -
Paul Sokolovsky -
Rob Cliffe -
Stephen J. Turnbull -
Steve Barnes -
Steven D'Aprano