[Python-Dev] Fwd: PEP: Frequently-requested additional features for the `unittest` module

[Olemis Lang]

>
 >  Abstract
 >  ========
 >
 >  This PEP proposes frequently-requested additions to the standard
 >  library `unittest` module that are natural extensions of its existing
 >  functionality.
 >

I had not seen this proposal... I will post a few comments.

 >
 >  Motivation
 >  ==========
 >
 >  The `unittest` module is functionally complete. Nevertheless, many
 >  users request and devise extensions to perform functions that are so
 >  common they deserve to be in the standard module.
 >
 >
 >  Specification
 >  =============
 >
 >  New condition tests
 >  -------------------
 >
 >  The following test methods will be added to the ``TestCase`` class.
 >  The function signature is part of this specification. The body is to
 >  be treated as a reference implementation only; any functionally
 >  identical implementation also meets this specification.
 >

...

 >
 >  The following test methods are also added. Their implementation in
 >  each case is simply the logical inverse of a corresponding method
 >  above.
 >

My question here is... should assert* (fail*) methods be included in a
 separate class or module? I think this is a good idea (at least in the
 long term) Take a look at the following thread [1] for further details

 >
 >  Simple invocation of test collection
 >  ------------------------------------
 >
 >  The following new functionality will be added to the `unittest`
 >  module. The function signature for ``run_tests`` is part of this
 >  specification; the implementation is to be considered a reference
 >  implementation only.
 >
 >  ::
 >
 >     def run_tests(
 >         tests,
 >         loader_class=TestLoader, suite_class=TestSuite,
 >         runner_class=TextTestRunner):
 >         """ Run a collection of tests with a test runner
 >
 >             :param tests:
 >                 A sequence of objects that can contain test cases:
 >                 modules, `TestSuite` instances, or `TestCase`
 >                 subclasses.
 >             :param loader_class:
 >                 The type of test loader to use for collecting tests
 >                 from the `tests` collection.
 >             :param suite_class:
 >                 The type of test suite to use for accumulating the
 >                 collected test cases.
 >             :param runner_class:
 >                 The type of test runner to instantiate for running the
 >                 collected test cases.
 >             :return:
 >                 None.
 >
 >             """
 >

I suppose that the reason for this function to exist is to ease test
 code (provide a shortcut or something), reuse some test code, and
 still allow different loading, running strategies. However, I dont
 think that passing (loader / suite / runner) classes in to this
 function is a good idea... IMO objects should be considered instead...

 Why? Mainly because the code assumes that all loaders are stateless.
 This may be true for unittest's and perhaps others, but this might not
 always be true. An example of statefull test loaders might be found in
 [2] (presented in [3]), and perhaps there other examples like this
 (even though I don't know them). This kind of code might restrain the
 use of this kind of loaders, and therefore might not be as reusable as
 should be.

 Nonetheless, you might think this is a very peculiar case and decide
 that this comment is irrelevant, but as far as I know, semantics of
 unittest TestLoader do not enforce that its subclasses ought to be
 stateless (although, I repeat, unittest test loaders are stateless).

 The same might be said for test runners... and they are more likely to
 be stateful.

 >         def iter_testcases_recursively(collection, loader):
 >             # Flatten and iterate over collection, generating
 >             # instances of TestCase
 >         loader = loader_class()
 >         suite = suite_class()
 >         for testcase in iter_testcases_recursively(tests, loader):
 >             suite.add_tests(testcase)
 >         runner = runner_class()
 >         runner.run(suite)

Besides... why is it necessary to flatten a hierarchy of test suites
 containing test cases? Why do you need to put them into another suite
 if the test loader already does this for us? Isn't it an extra
 overhead? Also there is a class attribute (`suiteClass`) in TestLoader
 which allows doing something like this.

 [1]    [Python-Dev] unittest Suggestions
        (http://mail.python.org/pipermail/python-dev/2008-August/081763.html)

 [2]    Module dutest
        (https://sourceforge.net/project/showfiles.php?group_id=220287&abmode=1)

 [3]    [Python-Dev] An OO API for doctest / unittest integration...
        (http://mail.python.org/pipermail/python-dev/2008-August/081761.html)

 --
 Regards,


 Olemis.