Art of Unit Testing

[David Goodger]

Paul Moore <gustav at morpheus.demon.co.uk> wrote on 2001-08-18 08:19:
> On Fri, 17 Aug 2001 11:32:49 -0700, Jeff Shannon <jeff at ccvcorp.com> wrote:
>> Well, the alternative argument could be that, if you don't need to setup and
>> teardown in between various tests, then they could be coded as subtests of a
>> single, larger test...  OTOH, if you *do* have high setup/teardown overhead,
>> which *does* require being redone for each test, then coding it each time
>> would be a pain.  Lumping subtests together is easier than multiple copies of
>> setup/teardown...
> 
> I'm not sure I agree. The long and short of it is that both cases can happen.
> But look at my example "for real" (sort of)
> 
> class testSimpleQueries(unittest.TestCase):
>     def setUp(self):
>         # May take seconds to execute...
>         self.connection = DB.Connect(connect_str)
>     def tearDown(self):
>         self.connection.Disconnect()
>     def exec_query(self, q):
>         "Trivial helper to run a query"
>         return self.connection.Execute(q)
>     def testQ1(self):
>         "Simple query"
>         q = "select 1 from dual"
>         assertEqual(1, self.exec_query(q))
>     def testQ2(self):
>         "Exception when no rows returned"
>         q = "select 1 from dual where 1=0"
>         assertRaises(DB.Error, self.exec_query(q))
>     # And 100 more trivial queries...
> 
> You get the idea. The setup costs a *lot* in relative terms of time (each test
> query takes, say, 0.01 second). You really want to only do that once. But you
> don't really want to code a single huge test - it destroys the reporting of
> the individual test docstrings by unittest.main(). What do you do?

The setUp/tearDown mechanism can ensure that each test is independent of any
state left over from any other test. If you know your tests won't leave any
state behind, you could put your DB.connect() call in
testSimpleQueries.__init__. (Don't forget to call unittest.TestCase.__init__
though.) Again, this assumes that none of the individual tests will change
the state of the connection. To close the connection, you could put the
self.connection.Disconnect() call in testSimpleQueries.__del__; but there's
no guarantee when (or even if?) this will be called.

It might be useful for unittest.py to support a
"globalSetUp"/"globalTearDown" mechanism. globalSetUp would be called once
at the initialization of the TestCase, before any tests are run. Unlike
overriding __init__, there would be no need for globalSetUp to call the
overridden method in the superclass. globalTearDown would be called after
all the tests have been run. The timing of the call to globalTearDown would
be well known, unlike __del__.

> I don't say it's not possible to code this - just that the documentation
> doesn't make it clear *how*.
> 
> Remember, I'm arguing that the common cases should be simple to code - not
> that it's not possible to do these things. That makes people more likely to
> code unit tests.
> 
> So really, I'm suggesting two things - (1) beef up the documentation of
> unittest.main(), and give it a section to itself ("How to code a simple series
> of unit tests"), and (2) if necessary, beef up unittest.main() to cater for
> such common cases (I don't necessarily think this is required).

Good suggestions. Unfortunately, active developers can't always read every
newsgroup posting, even though they may be relevant. (I've copied this post
to Steve Purcell, who's the head of the PyUnit project.) Might I suggest
that, if you'd like your ideas to survive longer than the lifetime of a
Usenet posting (roughly equivalent to the lifespan of a fruit fly), you (a)
submit them as feature requests to the PyUnit project at
http://sourceforge.net/projects/pyunit/ or to Python at
http://sourceforge.net/projects/python/, and perhaps even (b) work on the
documentation and code and submit some patches? Anybody can do #a, even if
they can't do #b.

-- 
David Goodger    dgoodger at users.sourceforge.net    Open-source projects:
 - Python Docstring Processing System: http://docstring.sourceforge.net
 - reStructuredText: http://structuredtext.sourceforge.net
 - The Go Tools Project: http://gotools.sourceforge.net