[Python-Dev] setUpClass and setUpModule in unittest

[R. David Murray]

On Wed, 10 Feb 2010 09:45:41 -0500, Olemis Lang <olemis at gmail.com> wrote:
> On Tue, Feb 9, 2010 at 5:34 PM, Holger Krekel <holger.krekel at gmail.com> wrote:
> > On Tue, Feb 9, 2010 at 10:57 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> >> Michael Foord <fuzzyman at voidspace.org.uk> writes:
> >>
> >>> The next 'big' change to unittest will (may?) be the introduction of
> >>> class and module level setUp and tearDown. This was discussed on
> >>> Python-ideas and Guido supported them. They can be useful but are also
> >>> very easy to abuse (too much shared state, monolithic test classes and
> >>> modules). Several authors of other Python testing frameworks spoke up
> >>> *against* them, but several *users* of test frameworks spoke up in
> >>> favour of them. ;-)
> >>
> >> I think the perceived need for these is from people trying to use the
> >> unittest API for test that are *not* unit tests.
> >>
> 
> Well the example I was talking about before is when some (critical)
> resource needed for unittesting requires a very, very heavy
> initialization process. I'll employ the most recent example (hope it
> doesn't look like too much biased, it's just to illustrate the whole
> picture ;o) which is unittests for a framework like Trac . In that
> case it is critical to have a Trac environment, a ready-to-use DB and
> backend, initialize the plugins cache by loading relevant plugins, run
> the actions specified by each IEnvironmentSetup participant, sometimes
> a ready to use repository (if testing code depending on Trac VCS API)
> and more ... Just considering these cases someone could :
> 
>   - Create a fake environment used as a stub
>   - But having a single global environment is not a good idea because
>      it would be very difficult to run multiple (independent) tests
>      concurrently (e.g. test multiple Trac plugins concurrently in a dedica=
> ted
>      CI environment). So an environment has to be started for every
>      test run and be as isolated as possible from other similar
>      stub environments
>   - The DB and backend can be replaced by using in-memory SQLite
>      connection
>   - Plugins cache and loading is mandatory as well running the actions
>      specified by each IEnvironmentSetup participant
>   - VCS can be mocked, but if it's needed it has to be initialized as well
> 
> And all this is needed to run *ANY* test of *ANY* kind (that includes
> unittests ;o) . I hope that, up to this point, you all are convinced

This doesn't sound very unit-testy, really.  It sounds like you are
operating at a rather high level (closer to integration testing).
As someone else said, I don't see anything wrong with using unittest
as a basis for doing that myself, but I don't think your example is a
clear example of wanting a setup and teardown that is executed once per
TestCase for tests that are more obviously what everyone would consider
"unit" tests.

I do have an example of that, though.  I have an external database
containing test data.  My unittests are generated on the fly so that each
generated test method pulls one set of test data from the database and
runs the appropriate checks that the package processes the data correctly.
(If you are curious, I'm testing email header parsing, and there are
a lot of different possible quirky headers that the parser needs to be
checked against)  Putting the test data in a database makes managing
the test data easier, and makes it available to other test frameworks
to reuse the data.

So, having the connection to the database set up once at TestCase start,
and closed at TestCase end, would make the most sense.  Currently there's
no way I know of to do that, so I open and close the database for every
unittest.  Fortunately it's an sqlite database, so the run time penalty
for doing that isn't prohibitive.

--
R. David Murray                                      www.bitdance.com