[Python-Dev] PEP: Frequently-requested additional features for the `unittest` module (version 0.5)

Ben Finney ben+python at benfinney.id.au
Thu Jul 17 01:14:48 CEST 2008

Significant changes: targeting Python 3.1, removal of separate
{lt,gt,le,ge} comparison tests, implementation of enhanced-information
failure message, reference to BDFL pronouncement.

I won't be working on this further; someone else should feel free to
champion this further if they wish.

:PEP:               XXX
:Title:             Frequently-requested additional features for the `unittest` module
:Version:           0.5
:Last-Modified:     2008-07-16
:Author:            Ben Finney <ben+python at benfinney.id.au>
:Status:            Draft
:Type:              Standards Track
:Content-Type:      test/x-rst
:Requires:          PEP XXX (Consolidating names in the `unittest` module)
:Created:           2008-07-14
:Python-Version:    3.1

..  contents::


This PEP proposes frequently-requested additions to the standard
library `unittest` module that are natural extensions of its existing


The `unittest` module is functionally complete. Nevertheless, many
users request and devise extensions to perform functions that are so
common they deserve to have a standard implementation.


New condition tests

The following test methods will be added to the ``TestCase`` class.
The function signature is part of this specification. The body is to
be treated as a reference implementation only; any functionally
identical implementation also meets this specification.


    import operator
    import re

    class TestCase(object):
        # …

        def assert_compare_true(op, first, second, msg=None):
            fail_detail = "%(first)r %(op)r %(second)r" % vars()
            if msg is None:
                msg = fail_detail
                msg = "%(fail_detail)s: %(msg)s" % vars()
            if not op(first, second):
                raise self.failure_exception(msg)

        def assert_in(container, member, msg=None):
            op = operator.__contains__
            self.assert_compare_true(op, container, member, msg)

        def assert_is(first, second, msg=None):
            op = operator.is_
            self.assert_compare_true(op, first, second, msg)

        def assert_members_equal(first, second, msg=None):
            op = operator.eq
            self.assert_compare_true(op, set(first), set(second), msg)

        def assert_sequence_equal(first, second, msg=None):
            op = operator.eq
            self.assert_compare_true(op, list(first), list(second), msg)

        def assert_raises_with_message_regex(
            exc_class, message_regex, callable_obj, *args, **kwargs):
            exc_name = exc_class.__name__
            message_pattern = re.compile(message_regex)
                callable_obj(*args, **kwargs)
            except exc_class, exc:
                exc_message = str(exc)
                if not message_pattern.match(exc_message):
                    msg = (
                        "%(exc_name)s raised"
                        " without message matching %(message_regex)r"
                        " (got message %(exc_message)r)"
                        ) % vars()
                    raise self.failure_exception(msg)
                msg = "%(exc_name)s not raised" % vars()
                raise self.failure_exception(msg)

The following test methods are also added. Their implementation in
each case is simply the logical inverse of a corresponding method


        def assert_compare_false(op, first, second, msg=None):
            # Logical inverse of assert_compare_true

        def assert_not_in(container, member, msg=None):
            # Logical inverse of assert_in

        def assert_is_not(first, second, msg=None)
            # Logical inverse of assert_is

        def assert_members_not_equal(first, second, msg=None)
            # Logical inverse of assert_members_equal

        def assert_sequence_not_equal(first, second, msg=None)
            # Logical inverse of assert_sequence_equal

Enhanced failure message for comparison tests

The comparison tests will change their behaviour such that the message
always, even if overridden with a specific message when called,
includes extra information:

* For both ``assert_equal`` and ``assert_not_equal``: the ``repr()``
  output of the objects that were compared. This matches the behaviour
  of ``assert_compare_true`` and ``assert_compare_false``, above.

* For ``assert_equal`` comparisons of ``basestring`` instances that
  are multi-line text: the output of ``diff`` comparing the two texts.

* For membership comparisons with ``assert_members_equal`` and
  ``assert_sequence_equal``: the ``repr()`` output of the members that
  were not equal in each collection. (This change is not done for the
  corresponding ``assert_*_not_equal`` tests, which only fail if the
  collection members are equal.)

Simple invocation of test collection

To allow simpler loading and running of test cases from an arbitrary
collection, the following new functionality will be added to the
`unittest` module. The function signatures are part of this
specification; the implementation is to be considered a reference
implementation only.


    class TestLoader(object):
        # …

        def load_tests_from_collection(self, collection):
            """ Return a suite of all test cases found in `collection`

                :param collection:
                    One of the following:
                    * a `TestSuite` instance
                    * a `TestCase` subclass
                    * a module
                    * an iterable producing any of these types
                    A `TestSuite` instance containing all the test
                    cases found recursively within `collection`.

            suite = None
            if isinstance(collection, TestSuite):
                suite = collection
            elif isinstance(collection, type):
                if issubclass(collection, TestCase):
                    suite = self.load_tests_from_test_case(collection)
            elif isinstance(collection, types.ModuleType):
                suite = self.load_tests_from_module(collection)
            elif hasattr(collection, '__iter__'):
                suite = self.suite_class()
                for item in collection:
                msg = "not a test collection: %(collection)r" % vars()
                raise TypeError(msg)
            return suite

    def run_tests(
        loader_class=TestLoader, runner_class=TextTestRunner):
        """ Run a collection of tests with a test runner

            :param tests:
                A test collection as defined by the `loader_class`
                method `load_tests_from_collection`.
            :param loader_class:
                The type of test loader to use for collecting tests
                from the `tests` collection.
            :param runner_class:
                The type of test runner to instantiate for running the
                collected test suite.

        loader = loader_class()
        suite = loader.load_tests_from_collection(tests)
        runner = runner_class()


BDFL pronouncement

The BDFL pronounced [#vanrossum-1]_ a set of boundaries within which
changes to the `unittest` module need to operate. This PEP may
currently violate some of those, making it currently unacceptable.

Names for logical-inverse tests

The simple pattern established by ``assert_foo`` having a logical
inverse named ``assert_not_foo`` sometimes results in gramatically
awkward names. The following names were chosen in exception of this
pattern, in the interest of the API names being more intuitive:

* ``assert_is_not``
* ``assert_members_not_equal``
* ``assert_sequence_not_equal``

Order of method parameters

The methods ``assert_in``, ``assert_not_in`` have the container as the
first parameter. This makes the grammatical object of the function
name come immediately after the function name: "Assert in
`container`". This matches the convention set by the existing
``assert_raises(exception, callable_obj, …)`` "(Assert the code
raises `exception`").

Set of additional tests

Test methods that were considered but failed to gain significant
support include:

* ``assert_less_than``, ``assert_greater_than``,
  ``assert_less_than_or_equal``, ``assert_greater_than_or_equal``, and
  logical inverses.

  These, and other less-used boolean comparison tests, can all be
  covered adequately with the new ``assert_compare_true`` and
  ``assert_compare_false`` methods with an appropriate comparison
  operator function.

Backwards Compatibility

This PEP proposes only additional features. There are no
backward-incompatible changes.

Reference Implementation

None yet.


This document is hereby placed in the public domain by its author.

..  [#vanrossum-1] http://mail.python.org/pipermail/python-dev/2008-July/081263.html

    Local Variables:
    mode: rst
    coding: utf-8
    vim: filetype=rst :

More information about the Python-Dev mailing list