[Tutor] writing effective unittests

[Steven D'Aprano]

On 03/01/13 07:43, Luke Thomas Mergner wrote:
> Hi,
>
> I am trying to learn a bit of test-driven programming using
>unittests and nosetests. I am having trouble finding resources
>that explain how to write effective tests. I am not a programmer
> or a student, so I do not have access to collegues or classes.

If you program, you are a programmer. And we are your colleagues :)


> I'm hoping that someone knows of a good tutorial that I've missed.
>  Based on the O'Reilly 2-Part introduction, I hope that learning to
>write tests will make my code better.


There are many resources on the web that talk about "Test Driven
Development". The basic idea is, you incrementally write the test first,
check that it fails, and then fix the code so the test passes. Then
write the next test, see that this one fails, then fix the code so that
it passes, and repeat until your code is done.

If you google for "Python test driven development tutorial", you will
find many resources:

https://duckduckgo.com/?q=python%20test%20driven%20development%20tutorial


Personally, I don't bother with the strict "write one test at a time"
rule, nor do I bother with "always write the test before the code". I am
quite happy to write a bunch of code first, and then write half a dozen
tests. The important part is to have the tests, not the order in which
they get written.

For example, suppose you have a function called "spam" that is supposed
to return a string:

def spam(n):
     """Return n lots of a yummy meat-like substance."""
     return ' '.join("spam")*n



So the first thing that I do is decide what sort of behaviour spam should
have. It should always return a string; it should only accept a single int
argument; if no argument is given, the result should be the same as
calling spam(1). So here are my first three unittests.


import unittest

class SpamTests(unittest.TestCase):
     # Tests related to the spam function.

     def testAlwaysReturnString(self):
         # Test that spam always returns a string.
         for n in (-5, -2, 0, 1, 2, 3, 15, 100):
             self.assertTrue(isinstance(spam(n), str))

     def testFailOnWrongArgumentCount(self):
         # Test that spam takes at most one argument.
         self.assertRaises(TypeError, spam, 1, 2)
         self.assertRaises(TypeError, spam, 1, 2, 3)

     def testDefaultArgIsOne(self):
         # Test that spam with no args is like spam(1).
         self.assertEqual(spam(), spam(1))



Notice that tests are not necessarily definitive. I haven't tested that
spam(n) returns a string for every imaginable integer n, because there
are too many. Instead, I just test a small, representative sample.

Likewise I haven't tested that spam() might succeed when given four
arguments, or forty-four. I trust that if it fails with two arguments,
and fails with three arguments, it will fail with more than three too.

Don't make the mistake of thinking that tests need to cover EVERYTHING
or else they are useless. 10% test coverage is better than nothing,
50% is better still, and 90% is better still. (100% is, of course,
impossible.)

At this point, I will run the unittests, and see whether they work or
not. I haven't actually done so, but I expect that testDefaultArgIsOne
will fail (to be precise, it will actually raise an exception). So I
then go back to my spam() function and add in a default argument:

def spam(n=1):
     ... # as above


Let's add some more tests to SpamTests:

     def testHasDocString(self):
         # Test that the spam function has a docstring.
         doc = spam.__doc__
         self.assertTrue(doc is not None)

     def testFailOnWrongArgumentType(self):
         # Test that the argument must not be a non-int.
         for bad_arg in (None, "hello", "42", 23.0, [], {}):
             self.assertRaises(TypeError, spam, bad_arg)

     def testReturnResult(self):
         # Test that spam returns the correct result.
         self.assertEqual(spam(1), "spam")
         self.assertEqual(spam(2), "spam spam")
         self.assertEqual(spam(3), "spam spam spam")
         self.assertEqual(spam(44), "spam spam spam spam")

     def testEmpty(self):
         # Test that spam returns an empty string when appropriate.
         for n in (0, -1, -2, -99):
             self.assertEqual(spam(n), "")


Then I run the tests. If any fail, I have to either fix the test (perhaps
the test is buggy) or fix the function.

Once those tests pass, I might decide that I'm finished -- I have enough
tests for spam, and can go on to the next part of my code. Or I might
decide that I don't actually like the behaviour of spam as it is now. For
instance, I might change the function to this:


def spam(n=3):
     """Return n lots of a yummy meat-like substance."""
     if n < 0:
         return "No spam for YOU!!!"
     return ' '.join("spam")*n


Now I run my tests again. I expect that now two tests will fail. Can you
see which ones? So I modify the tests, or add new tests as needed, and
continue until I'm happy with the results. And then move on to the next
part of my code.


The important thing here is that there process continually goes back and
forth between the tests and the main code. Adding new tests reveals bugs
in the code, or missing functionality, which you then fix, then write
more tests, until you can no longer think of any more tests.




-- 
Steven