[Python-Dev] PEP: Frequently-requested additional features for the `unittest` module

Ben Finney ben+python at benfinney.id.au
Wed Jul 16 07:48:09 CEST 2008

Significant changes: Add a new 'TestLoader.load_tests_from_collection'
method, with full reference implementation. This makes the 'run_tests'
reference implementation straightforward.

:PEP:           XXX
:Title:         Frequently-requested additional features for the `unittest` module
:Version:       0.4
:Last-Modified: 2008-07-16
:Author:        Ben Finney <ben+python at benfinney.id.au>
:Status:        Draft
:Type:          Standards Track
:Content-Type:  test/x-rst
:Requires:      PEP XXX (Consolidating names in the `unittest` module)
:Created:       2008-07-14

..  contents::


This PEP proposes frequently-requested additions to the standard
library `unittest` module that are natural extensions of its existing


The `unittest` module is functionally complete. Nevertheless, many
users request and devise extensions to perform functions that are so
common they deserve to have a standard implementation.


New condition tests

The following test methods will be added to the ``TestCase`` class.
The function signature is part of this specification. The body is to
be treated as a reference implementation only; any functionally
identical implementation also meets this specification.


    import operator
    import re

    class TestCase(object):
        # …

        def assert_compare_true(op, first, second, msg=None):
            if msg is None:
                msg = "%(first)r %(op)r %(second)" % vars()
            if not op(first, second):
                raise self.failure_exception(msg)

        def assert_in(container, member, msg=None):
            op = operator.__contains__
            self.assert_compare_true(op, container, member, msg)

        def assert_is(first, second, msg=None):
            op = operator.is_
            self.assert_compare_true(op, first, second, msg)

        def assert_less_than(first, second, msg=None):
            op = operator.lt
            self.assert_compare_true(op, first, second, msg)

        def assert_greater_than(first, second, msg=None):
            op = operator.gt
            self.assert_compare_true(op, first, second, msg)

        def assert_less_than_or_equal(first, second, msg=None):
            op = operator.le
            self.assert_compare_true(op, first, second, msg)

        def assert_greater_than_or_equal(first, second, msg=None):
            op = operator.ge
            self.assert_compare_true(op, first, second, msg)

        def assert_members_equal(first, second, msg=None):
            self.assert_equal(set(first), set(second), msg)

        def assert_sequence_equal(first, second, msg=None):
            self.assert_equal(list(first), list(second), msg)

        def assert_raises_with_message_regex(
            exc_class, message_regex, callable_obj, *args, **kwargs):
            exc_name = exc_class.__name__
            message_pattern = re.compile(message_regex)
                callable_obj(*args, **kwargs)
            except exc_class, exc:
                exc_message = str(exc)
                if not message_pattern.match(exc_message):
                    msg = (
                        "%(exc_name)s raised"
                        " without message matching %(message_regex)r"
                        " (got message %(exc_message)r)"
                        ) % vars()
                    raise self.failure_exception(msg)
                msg = "%(exc_name)s not raised" % vars()
                raise self.failure_exception(msg)

The following test methods are also added. Their implementation in
each case is simply the logical inverse of a corresponding method


        def assert_compare_false(op, first, second, msg=None):
            # Logical inverse of assert_compare_true

        def assert_not_in(container, member, msg=None):
            # Logical inverse of assert_in

        def assert_is_not(first, second, msg=None)
            # Logical inverse of assert_is

        def assert_not_less_than(first, second, msg=None)
            # Logical inverse of assert_less_than

        def assert_not_greater_than(first, second, msg=None)
            # Logical inverse of assert_greater_than

        def assert_not_less_than_or_equal(first, second, msg=None)
            # Logical inverse of assert_less_than_or_equal

        def assert_not_greater_than_or_equal(first, second, msg=None)
            # Logical inverse of assert_greater_than_or_equal

        def assert_members_not_equal(first, second, msg=None)
            # Logical inverse of assert_members_equal

        def assert_sequence_not_equal(first, second, msg=None)
            # Logical inverse of assert_sequence_equal

Enhanced failure message for equality tests

The equality tests will change their behaviour such that the message
always, even if overridden with a specific message when called,
includes extra information:

* For both ``assert_equal`` and ``assert_not_equal``: the ``repr()``
  output of the objects that were compared.

* For ``assert_equal`` comparisons of ``basestring`` instances that
  are multi-line text: the output of ``diff`` comparing the two texts.

* For membership comparisons with ``assert_*_equal``: the ``repr()``
  output of the members that were not equal in each collection. (This
  change is not done for the corresponding ``assert_*_not_equal``
  tests, which only fail if the collection members are equal.)

Simple invocation of test collection

To allow simpler loading and running of test cases from an arbitrary
collection, the following new functionality will be added to the
`unittest` module. The function signatures are part of this
specification; the implementation is to be considered a reference
implementation only.


    class TestLoader(object):
        # …

        def load_tests_from_collection(self, collection):
            """ Return a suite of all test cases found in `collection`

                :param collection:
                    One of the following:
                    * a `TestSuite` instance
                    * a `TestCase` subclass
                    * a module
                    * an iterable producing any of these types
                    A `TestSuite` instance containing all the test
                    cases found recursively within `collection`.

            suite = None
            if isinstance(collection, TestSuite):
                suite = collection
            elif isinstance(collection, type):
                if issubclass(collection, TestCase):
                    suite = self.load_tests_from_test_case(collection)
            elif isinstance(collection, types.ModuleType):
                suite = self.load_tests_from_module(collection)
            elif hasattr(collection, '__iter__'):
                suite = self.suite_class()
                for item in collection:
                msg = "not a test collection: %(collection)r" % vars()
                raise TypeError(msg)
            return suite

    def run_tests(
        loader_class=TestLoader, runner_class=TextTestRunner):
        """ Run a collection of tests with a test runner

            :param tests:
                A test collection as defined by the `loader_class`
                method `load_tests_from_collection`.
            :param loader_class:
                The type of test loader to use for collecting tests
                from the `tests` collection.
            :param runner_class:
                The type of test runner to instantiate for running the
                collected test suite.

        loader = loader_class()
        suite = loader.load_tests_from_collection(tests)
        runner = runner_class()


Names for logical-inverse tests

The simple pattern established by ``assert_foo`` having a logical
inverse named ``assert_not_foo`` sometimes results in gramatically
awkward names. The following names were chosen in exception of this
pattern, in the interest of the API names being more intuitive:

* ``assert_is_not``
* ``assert_members_not_equal``
* ``assert_sequence_not_equal``

Order of method parameters

The methods ``assert_in``, ``assert_not_in`` have the container as the
first parameter. This makes the grammatical object of the function
name come immediately after the function name: "Assert in
`container`". This matches the convention set by the existing
``assert_raises(exception, callable_obj, …)`` "(Assert the code
raises `exception`").

Backwards Compatibility

This PEP proposes only additional features. There are no
backward-incompatible changes.

Reference Implementation

None yet.


This document is hereby placed in the public domain by its author.

    Local Variables:
    mode: rst
    coding: utf-8
    vim: filetype=rst :

More information about the Python-Dev mailing list