[Python-Dev] setUpClass and setUpModule in unittest

[Guido van Rossum]

On Tue, Feb 9, 2010 at 8:42 AM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> The next 'big' change to unittest will (may?) be the introduction of class
> and module level setUp and tearDown. This was discussed on Python-ideas and
> Guido supported them. They can be useful but are also very easy to abuse
> (too much shared state, monolithic test classes and modules). Several
> authors of other Python testing frameworks spoke up *against* them, but
> several *users* of test frameworks spoke up in favour of them. ;-)

Hi Michael,

I have skimmed this thread (hence this reply to the first rather than
the last message), but in general I am baffled by the hostility of
testing framework developers towards their users. The arguments
against class- and module-level seUp/tearDown functions seems to be
inspired by religion or ideology more than by the zen of Python. What
happened to Practicality Beats Purity?

The potential for abuse in and of itself should not be an argument
against a feature; it must always be weighed against the advantages.

The argument that a unittest framework shouldn't be "abused" for
regression tests (or integration tests, or whatever) is also bizarre
to my mind. Surely if a testing framework applies to multiple kinds of
testing that's a good thing, not something to be frowned upon?

There are several alternative testing frameworks available outside the
standard library. The provide useful competition with the stlib's
unittest and doctest modules, and useful inspiration for potential new
features. They also, by and large, evolve much faster than a stdlib
module ever could, and including anyone of these in the stdlib might
well be the death of it (just as unittest has evolved much slower
since it was included).

But unittest *is* still evolving, and there is no reason not to keep
adding features along the lines of your module/class setUp/tearDown
proposal (or extra assertions like assertListEqual, which I am happy
to see has been added).

On the other hand, I think we should be careful to extend unittest in
a consistent way. I shuddered at earlier proposals (on python-ideas)
to name the new functions (variations of) set_up and tear_down "to
conform with PEP 8" (this would actually have violated that PEP, which
explicitly prefers local consistency over global consistency).

I also think that using a with-statement or a decorator to indicate
the scope of setUp/tearDown operations flies in the face of the
existing "style" of the unittest module (um, package, I know :-),
which is based on defining setUp and tearDown methods with specific
semantics.

Regarding the objection that setUp/tearDown for classes would run into
issues with subclassing, I propose to let the standard semantics of
subclasses do their job. Thus a subclass that overrides setUpClass or
tearDownClass is responsible for calling the base class's setUpClass
and tearDownClass (and the TestCase base class should provide empty
versions of both). The testrunner should only call setUpClass and
tearDownClass for classes that have at least one test that is
selected.

Yes, this would mean that if a base class has a test method and a
setUpClass (and tearDownClass) method and a subclass also has a test
method and overrides setUpClass (and/or tearDown), the base class's
setUpClass and tearDown may be called twice. What's the big deal? If
setUpClass and tearDownClass are written properly they should support
this. If this behavior is undesired in a particular case, maybe what
was really meant were module-level setUp and tearDown, or the class
structure should be rearranged.

Anyway, Michael, thanks for getting this started -- I support your
attempts to improve the unittest package and am writing in the hope
that the discussion will soon converge and patches whipped up.

-- 
--Guido van Rossum (python.org/~guido)