[Python-Dev] "if __name__ == '__main__'" at the bottom of python unittest files

[Terry Reedy]

On 5/1/2019 9:30 AM, Chris Withers wrote:

> Agreed, but my focus here is to get to 100% for mock so that it's clear 
> that all the code is there for a reason; mock is very complicated by 
> necessity, and having examples of why code needs to be there is what I'm 
> aiming for most of all.

I agree that complete 100.000% test coverage is a nice ideal, but 
sometimes the last percent can take hours to accomplish, if it is indeed 
sensibly possible.  (If mock has no OS dependencies, then it may be for 
mock.)  It is up to the individual developer to decide what is the 
priority to spend development time on.

>> Is it really that difficult to simply tell coverage to ignore them? I
>> thought someone had already pointed to a coveragerc file that let you
>> do this.
> 
> It would be if the cpython repo had a coveragerc, but it does not.

I have asked more that once what .coveragerc file is being used by CI 
and whether we can somehow properly customize it for CPython.  I have 
not gotten an answer.

The devguide chapter (5) on coverage is deficient in not even mentioning 
customization.

> People maintaining their own ad-hoc coverage configs seems like a pretty 
> bad idea.

I would prefer not having to do that.  But it is better than always 
getting bogus numbers.  At least *I* can determine the real single-file 
test coverage for idlelib files (on Windows), even if the public 
coverage reports are misleading.  Unless I forget, I record it on the 
first line of text_xxx files.

> Right, that's by gut feeling here: I don't want people encountering this 
> mock codebase to have to wonder whether they should be running the tests 
> using these blocks versus the way described in the dev guide,

The devguide describes the dependable but clumsy way to run tests from a 
command line.  In my opinion, it is the worst way when one is editing a 
particular file. It is *much* easier to hit one key (F5 in IDLE) in an 
editor than to switch to a terminal window and type something like

   > python -m unittest idlelib.idle_test.test_configdialog

It is much better to have a SyntaxError marked in the editor than 
displayed in a terminal or shell.  Live tracebacks in an IDE, that can 
jump to exception locations in an editor are better than a dead 
traceback in a terminal window.

With IDLE there is also the issue that automated unittests cannot 
completely replace human action and judgment.  However, displaying 
isolated dialogs for human interaction can be automated.  The 'if main' 
blocks for dialog modules do so.  For example, for configdialog:

if __name__ == '__main__':
     from unittest import main
     main('idlelib.idle_test.test_configdialog', verbosity=2, exit=False)

     from idlelib.idle_test.htest import run
     run(ConfigDialog)

One can either close the box or check the visual appearance and live 
behavior of the dialog.  In this case, running the class is sufficient. 
For other modules, additional setup code is needed, which should be 
excluded from coverage.

> and stressing about what the differences might be, when there aren't any...

Unittest users should know that it has both code and command line APIs.
The devguide should mention running tests from code with main or refer 
to the appropriate section of the unittest doc.  It should at least 
document the use of

if __name__ == '__main__':
     unittest.main(verbosity=2)

in test_xxx modules.


-- 
Terry Jan Reedy