Exhaustive Unit Testing

[Steven D'Aprano]

On Wed, 26 Nov 2008 18:54:52 -0800, Emanuele D'Arrigo wrote:

> Hi everybody,
> 
> another question on unit testing, admittedly not necessarily a python-
> specific one...
> 
> I have a class method about 60 lines long (*) and due to some 9 non-
> trivial IFs statements (3 and 2 of which nested) the number of possible
> paths the program flow can take is uncomfortably large, each path
> characterized by an uncomfortably long list of careful turns to take.

Nine non-trivial if statements, some of which are nested... uncomfortably 
large... uncomfortably long...  these are all warning signs that your 
method is a monster. 

I would say the obvious solution is to refactor that method into multiple 
simpler methods, then call those. Not only does that make maintaining 
that class method easier, but you can also unit-test each method 
individually.

With nine if-statements, you have 512 program paths that need testing. 
(In practice, there may be fewer because some of the statements are 
nested, and presumably some paths will be mutually exclusive.) Refactor 
the if...else blocks into methods, and you might have as many as 18 paths 
to test. So in theory you should reduce the number of unit-tests 
significantly. 


> Given that all paths have large portions of them overlapping with each
> other, is there a strategy that would allow me to collapse the many
> cases into only a few representative ones?

That would be impossible to answer without knowing what the method 
specifically does and how the many cases interact with each other.


> I can sense that exhaustively testing all paths is probably overkill.

I don't agree. If you don't test all the paths, then by definition you 
have program paths which have never been tested. Unless those paths are 
so trivially simple that you can see that they must be correct just by 
looking at the code, then the chances are very high that they will hide 
bugs.


> On
> the other hand just going through all nodes and edges between the nodes
> -at least once- is probably not enough. Is there any technique to find
> the right(ish) middle?

Refactor until your code is simple enough to unit-test effectively, then 
unit-test effectively.



-- 
Steven