On 5/1/2019 9:30 AM, Chris Withers wrote:
Agreed, but my focus here is to get to 100% for mock so that it's clear that all the code is there for a reason; mock is very complicated by necessity, and having examples of why code needs to be there is what I'm aiming for most of all.
I agree that complete 100.000% test coverage is a nice ideal, but sometimes the last percent can take hours to accomplish, if it is indeed sensibly possible. (If mock has no OS dependencies, then it may be for mock.) It is up to the individual developer to decide what is the priority to spend development time on.
Is it really that difficult to simply tell coverage to ignore them? I thought someone had already pointed to a coveragerc file that let you do this.
It would be if the cpython repo had a coveragerc, but it does not.
I have asked more that once what .coveragerc file is being used by CI and whether we can somehow properly customize it for CPython. I have not gotten an answer. The devguide chapter (5) on coverage is deficient in not even mentioning customization.
People maintaining their own ad-hoc coverage configs seems like a pretty bad idea.
I would prefer not having to do that. But it is better than always getting bogus numbers. At least *I* can determine the real single-file test coverage for idlelib files (on Windows), even if the public coverage reports are misleading. Unless I forget, I record it on the first line of text_xxx files.
Right, that's by gut feeling here: I don't want people encountering this mock codebase to have to wonder whether they should be running the tests using these blocks versus the way described in the dev guide,
The devguide describes the dependable but clumsy way to run tests from a command line. In my opinion, it is the worst way when one is editing a particular file. It is *much* easier to hit one key (F5 in IDLE) in an editor than to switch to a terminal window and type something like
python -m unittest idlelib.idle_test.test_configdialog
It is much better to have a SyntaxError marked in the editor than displayed in a terminal or shell. Live tracebacks in an IDE, that can jump to exception locations in an editor are better than a dead traceback in a terminal window. With IDLE there is also the issue that automated unittests cannot completely replace human action and judgment. However, displaying isolated dialogs for human interaction can be automated. The 'if main' blocks for dialog modules do so. For example, for configdialog: if __name__ == '__main__': from unittest import main main('idlelib.idle_test.test_configdialog', verbosity=2, exit=False) from idlelib.idle_test.htest import run run(ConfigDialog) One can either close the box or check the visual appearance and live behavior of the dialog. In this case, running the class is sufficient. For other modules, additional setup code is needed, which should be excluded from coverage.
and stressing about what the differences might be, when there aren't any...
Unittest users should know that it has both code and command line APIs. The devguide should mention running tests from code with main or refer to the appropriate section of the unittest doc. It should at least document the use of if __name__ == '__main__': unittest.main(verbosity=2) in test_xxx modules. -- Terry Jan Reedy