<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
</head>
<body bgcolor="#ffffff" text="#000000">
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
Hello all,<br>
<br>
unittest has seen quite a few new features since Python 2.6. For those
of you who might have missed the announcements in other places,
unittest2 is a backport of the new features in unittest to work with
Python 2.4-2.6. It is already being used for the development of
distutils2:<br>
<br>
<a href="http://pypi.python.org/pypi/unittest2/" target="_blank">http://pypi.python.org/pypi/<wbr>unittest2/</a><br>
<br>
In other news, class and module fixtures (setUpClass / tearDownClass /
setUpModule / tearDownModule) are now implemented in unittest (in trunk
- not yet merged to py3k). These features are tested but I'm sure there
are some lurking bugs or oddities, so please try them out. I have not
yet added documentation for them; I'll pull it out from this email as a
starting point.<br>
<br>
I'd rather this thread didn't become *another* debate on the merit of
these features, but perhaps that is too much to hope for.<br>
<br>
Below are some notes on how class and module fixtures work.<br>
<br>
Class and module level fixtures are implemented in unittest.TestSuite.
When the test suite encounters a test from a new class then <span
class="il">tearDownClass</span> from the previous class (if there is
one) is called, followed by <span class="il">setUpClass</span> from
the new class.<br>
<br>
Similarly if a test is from a different module from the previous test
then tearDownModule from the previous module is run, followed by
setUpModule from the new module.<br>
<br>
After all the tests have run the final <span class="il">tearDownClass</span>
and tearDownModule are run.<br>
<br>
Note that shared fixtures do not play well with [potential] features
like test parallelization and they break test isolation. They should be
used with care.<br>
<br>
<br>
<span class="il">setUpClass</span> and <span class="il">tearDownClass</span><br>
------------------------------<wbr>--------<br>
<br>
These must be implemented as class methods.<br>
<br>
import unittest<br>
<br>
class Test(unittest.TestCase):<br>
@classmethod<br>
def <span class="il">setUpClass</span>(cls):<br>
cls._connection = createExpensiveConnectionObjec<wbr>t()<br>
<br>
@classmethod<br>
def <span class="il">tearDownClass</span>(cls):<br>
cls._connection.destroy()<br>
<br>
If you want the <span class="il">setUpClass</span> and <span
class="il">tearDownClass</span> on base classes called then you must
call up to them yourself. The implementations in unittest2.TestCase are
empty.<br>
<br>
If an exception is raised during a <span class="il">setUpClass</span>
then the tests in the class are not run and the <span class="il">tearDownClass</span>
is not run. Skipped classes will not have <span class="il">setUpClass</span>
or <span class="il">tearDownClass</span> run.<br>
<br>
A <span class="il">setUpClass</span> that raises a unittest2.SkipTest
exception will currently be reported as an error rather than a skip
(although the effect is the same). This will be fixed at some point in
the future.<br>
<br>
<br>
setUpModule and tearDownModule<br>
------------------------------<wbr>--------------<br>
<br>
These should be implemented as functions.<br>
<br>
def setUpModule():<br>
createConnection()<br>
<br>
def tearDownModule():<br>
closeConnection()<br>
<br>
If an exception is raised in a setUpModule then none of the tests in
the module will be run and the tearDownModule will not be run.<br>
<br>
A setUpModule that raises a unittest.SkipTest exception will currently
be reported as an error rather than a skip (although the effect is the
same). This will be fixed at some point in the future.<br>
<br>
<br>
The Gory Details<br>
----------------------<br>
<br>
The default ordering of tests created by the unittest test loaders is
to group all tests from the same modules and classes together. This
will lead to <span class="il">setUpClass</span> / setUpModule (etc)
being called exactly once per class and module. If you randomize the
order, so that tests from different modules and classes are adjacent to
each other, then these shared fixture functions may be called multiple
times in a single test run.<br>
<br>
This particular issue was a point of some disussion. Shared fixtures
are not intended to work with suites with non-standard (incompatible)
ordering (a BaseTestSuite still exists for frameworks that don't want
to support shared fixtures), so this is already a pathological
situation. There are a couple of things unittest could do in this
situation:<br>
<br>
1) Work out the best order to run the setUp and tearDowns, based on
the position of the first and last test from each class / module in the
suite.<br>
2) Run setUp and tearDowns appropriately if consecutive tests are
from different classes and modules.<br>
<br>
If your setUps do unrelated things then strategy 1 will be the best
one, but (so long as your setups are repeatable) 2 will be fine just
slower. If your setups do related things, for example pushing large
test datasets into the same database, then having multiple setups
active before the corresponding teardowns have been run will actually
do the wrong thing and your tests will be broken. In addition strategy
1 relies on unrolling the test suite and introspecting all tests prior
to the run, not all custom test suites implementations support this.<br>
<br>
Given that strategy 2 was substantially easier to implement, and less
likely to cause broken test runs, that was what I went with. It is of
course possible to add alternative strategies, plus the metadata to
support them, if there proves to be a substantial need for this. One of
the major use cases is for test order randomization. It would be easy
to add a 'randomize' method to the suite that randomizes the order of
test modules in a suite, the order of test classes within modules and
the order of tests within a class. This would allow for tests to be run
in a random order whilst remaining compatible with shared fixtures.<br>
<br>
If there are any exceptions raised during one of the shared fixture
functions the test is reported as an error. Because there is no
corresponding test instance an _ErrorHolder object (that has the same
interface as a TestCase) is created to represent the error. If you are
just using the standard unittest test runner then this detail doesn't
matter, but if you are a framework author it may be relevant.<br>
<br>
<br>
<br>
I have a few minor features I'd like to add to unittest before 2.7 beta
- standard out capturing, fail fast and ctrl-c handling (first ctrl-c
will wait for current test to exit, second will force an exit - results
will still be reported). After that the *really* big problem for
unittest is the extensibility story. <br>
<br>
I'll be working on extensibility points for unittest that allow people
to implement extensions without having to override functionality. This
will take a fair bit of discussion and experimentation, so I will
probably trial this in unittest2 first. It would be nice to have
something in place for the release of 3.2, but that may be ambitious.<br>
<br>
All the best,<br>
<br>
Michael Foord<br>
<font color="#888888"></font><br>
<pre class="moz-signature" cols="72">--
<a class="moz-txt-link-freetext" href="http://www.ironpythoninaction.com/">http://www.ironpythoninaction.com/</a>
<a class="moz-txt-link-freetext" href="http://www.voidspace.org.uk/blog">http://www.voidspace.org.uk/blog</a>
READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.
</pre>
</body>
</html>