PEP 299 and unit testing

Ben Finney ben at benfinney.id.au
Mon Oct 29 02:56:11 CET 2007


Howdy all,

PEP 299 <URL:http://www.python.org/dev/peps/pep-0299> details an
enhancement for entry points to Python programs: a module attribute
(named '__main__') that will be automatically called if the module is
run as a program.

The PEP has status "Rejected", citing backward-compatibility issues,
and Guido's pronouncement that "It's not worth the change (in docs,
user habits, etc.) and there's nothing particularly broken."

I don't deny the backward-compatibility issues in the cited
discussion, but I'd like to point out one thing that is broken by
this: unit testing of program modules.


Unit tests need to import a module and introspectively test small
units from the module to verify their behaviour in isolation. The
boundary of a unit test is the code that's actually in the module
under test: any functional code in that module needs to be tested by
the module's unit test, any code not in that module is outside the
scope of that unit test module.

The logical extension of this is to put *all* functional code into
discrete units, including the "main line" code that gets executed when
the module is run as a program. This leads to code of the type
discussed in PEP 299:

    def main(argv):
        """ Do the main stuff of this program """
        parse_commandline(argv)
        try:
            do_interesting_things()
        except SystemExit, e:
            exitcode = e.code
        return exitcode

    if __name__ == "__main__":
        import sys
        exitcode = main(sys.argv)
        sys.exit(exitcode)

This allows the module's 'main' function to be called as a discrete
unit from the unit test module; the unit test passes in 'argv' as
desired, and fakes out other units that aren't being tested.

What it doesn't allow is for the testing of the 'if __name__ ==
"__main__":' clause itself. No matter how simple we make that, it's
still functional code that can contain errors, be they obvious or
subtle; yet it's code that *can't* be touched by the unit test (by
design, it doesn't execute when the module is imported), leading to
errors that won't be caught as early or easily as they might.

So, I'd argue that "nothing particularly broken" isn't true: unit
testing is flawed in this scenario. It means that even the simple
metric of statement-level test coverage can't ever get to 100%, which
is a problem since it defeats a simple goal of "get all functional
code covered by unit tests".


On the other hand, if PEP 299 *were* implemented (and the
backward-compatibility issues solved), the above could be written as:

    def __main__(argv):
        """ Do the main stuff of this program """
        parse_commandline(argv)
        try:
            do_interesting_things()
        except SystemExit, e:
            exitcode = e.code
        return exitcode

with no module-level 'if __name__' test at all, and therefore no
functional code unreachable by the unit test module. The effect of the
program is the same, but the invocation of the '__main__' function
isn't left to be implemented in every single program, separately and
subject to error in every case. Instead, it becomes part of the
*external* environment of the module, and is trivially outside the
scope of a unit test module for that program.

-- 
 \          "What I have to do is see, at any rate, that I do not lend |
  `\   myself to the wrong which I condemn."  -- Henry Thoreau, _Civil |
_o__)                                                    Disobedience_ |
Ben Finney



More information about the Python-list mailing list