[Tutor] writing effective unittests

[Peter Otten]

Steven D'Aprano wrote:

> Notice that tests are not necessarily definitive. I haven't tested that
> spam(n) returns a string for every imaginable integer n, because there
> are too many. Instead, I just test a small, representative sample.
> 
> Likewise I haven't tested that spam() might succeed when given four
> arguments, or forty-four. I trust that if it fails with two arguments,
> and fails with three arguments, it will fail with more than three too.
> 
> Don't make the mistake of thinking that tests need to cover EVERYTHING
> or else they are useless. 10% test coverage is better than nothing,
> 50% is better still, and 90% is better still. (100% is, of course,
> impossible.)

There's no way you can test even 10% of the possible inputs for even the 
simple

def mul(a, b):
    return a * b

as there is an infinite number of such inputs. 

The more common and less ambitious definition of coverage is 

"What portion of your code is executed by your tests? 

There's an excellent tool to answer that question: 

http://nedbatchelder.com/code/coverage/

It reports executed lines, so problematic code like

x = 1 if True else undefined

will be reported as 100% coverage, but it is still a big help in improving 
your tests.

> Let's add some more tests to SpamTests:
> 
>      def testHasDocString(self):
>          # Test that the spam function has a docstring.
>          doc = spam.__doc__
>          self.assertTrue(doc is not None)

You should rather run a tool like pylint to find missing docstrings. 

> The important thing here is that there process continually goes back and
> forth between the tests and the main code. Adding new tests reveals bugs
> in the code, or missing functionality, which you then fix, then write
> more tests, until you can no longer think of any more tests.
 
If you take this advice seriously you will soon rewrite your application 
code to make it easier to test it. At least for me the impact of unittests 
was larger than I had expected. 

I really should do more of them...