Hello all, unittest has seen quite a few new features since Python 2.6. For those of you who might have missed the announcements in other places, unittest2 is a backport of the new features in unittest to work with Python 2.4-2.6. It is already being used for the development of distutils2: http://pypi.python.org/pypi/unittest2/ <http://pypi.python.org/pypi/unittest2/> In other news, class and module fixtures (setUpClass / tearDownClass / setUpModule / tearDownModule) are now implemented in unittest (in trunk - not yet merged to py3k). These features are tested but I'm sure there are some lurking bugs or oddities, so please try them out. I have not yet added documentation for them; I'll pull it out from this email as a starting point. I'd rather this thread didn't become *another* debate on the merit of these features, but perhaps that is too much to hope for. Below are some notes on how class and module fixtures work. Class and module level fixtures are implemented in unittest.TestSuite. When the test suite encounters a test from a new class then tearDownClass from the previous class (if there is one) is called, followed by setUpClass from the new class. Similarly if a test is from a different module from the previous test then tearDownModule from the previous module is run, followed by setUpModule from the new module. After all the tests have run the final tearDownClass and tearDownModule are run. Note that shared fixtures do not play well with [potential] features like test parallelization and they break test isolation. They should be used with care. setUpClass and tearDownClass -------------------------------------- These must be implemented as class methods. import unittest class Test(unittest.TestCase): @classmethod def setUpClass(cls): cls._connection = createExpensiveConnectionObject() @classmethod def tearDownClass(cls): cls._connection.destroy() If you want the setUpClass and tearDownClass on base classes called then you must call up to them yourself. The implementations in unittest2.TestCase are empty. If an exception is raised during a setUpClass then the tests in the class are not run and the tearDownClass is not run. Skipped classes will not have setUpClass or tearDownClass run. A setUpClass that raises a unittest2.SkipTest exception will currently be reported as an error rather than a skip (although the effect is the same). This will be fixed at some point in the future. setUpModule and tearDownModule -------------------------------------------- These should be implemented as functions. def setUpModule(): createConnection() def tearDownModule(): closeConnection() If an exception is raised in a setUpModule then none of the tests in the module will be run and the tearDownModule will not be run. A setUpModule that raises a unittest.SkipTest exception will currently be reported as an error rather than a skip (although the effect is the same). This will be fixed at some point in the future. The Gory Details ---------------------- The default ordering of tests created by the unittest test loaders is to group all tests from the same modules and classes together. This will lead to setUpClass / setUpModule (etc) being called exactly once per class and module. If you randomize the order, so that tests from different modules and classes are adjacent to each other, then these shared fixture functions may be called multiple times in a single test run. This particular issue was a point of some disussion. Shared fixtures are not intended to work with suites with non-standard (incompatible) ordering (a BaseTestSuite still exists for frameworks that don't want to support shared fixtures), so this is already a pathological situation. There are a couple of things unittest could do in this situation: 1) Work out the best order to run the setUp and tearDowns, based on the position of the first and last test from each class / module in the suite. 2) Run setUp and tearDowns appropriately if consecutive tests are from different classes and modules. If your setUps do unrelated things then strategy 1 will be the best one, but (so long as your setups are repeatable) 2 will be fine just slower. If your setups do related things, for example pushing large test datasets into the same database, then having multiple setups active before the corresponding teardowns have been run will actually do the wrong thing and your tests will be broken. In addition strategy 1 relies on unrolling the test suite and introspecting all tests prior to the run, not all custom test suites implementations support this. Given that strategy 2 was substantially easier to implement, and less likely to cause broken test runs, that was what I went with. It is of course possible to add alternative strategies, plus the metadata to support them, if there proves to be a substantial need for this. One of the major use cases is for test order randomization. It would be easy to add a 'randomize' method to the suite that randomizes the order of test modules in a suite, the order of test classes within modules and the order of tests within a class. This would allow for tests to be run in a random order whilst remaining compatible with shared fixtures. If there are any exceptions raised during one of the shared fixture functions the test is reported as an error. Because there is no corresponding test instance an _ErrorHolder object (that has the same interface as a TestCase) is created to represent the error. If you are just using the standard unittest test runner then this detail doesn't matter, but if you are a framework author it may be relevant. I have a few minor features I'd like to add to unittest before 2.7 beta - standard out capturing, fail fast and ctrl-c handling (first ctrl-c will wait for current test to exit, second will force an exit - results will still be reported). After that the *really* big problem for unittest is the extensibility story. I'll be working on extensibility points for unittest that allow people to implement extensions without having to override functionality. This will take a fair bit of discussion and experimentation, so I will probably trial this in unittest2 first. It would be nice to have something in place for the release of 3.2, but that may be ambitious. All the best, Michael Foord -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.