When is unit-testing bad? [was: Re: does lack of type...]

[Christopher Blunck]

Well put.

The argument for unit testing that I always use goes something like
this:

You're assigned a task.  Part of that task involves requirement
investigation, and task comprehension.  Before you build a widget, you
must first understand what you're trying to build, who you are
building it for, and why they want it built.  The next more tangible
part is the design of the widget, for which there are various
approaches outside the realm of this discussion.  After design, an
implement and test activity begins.  The order of this is not
particularly significant to me, but I do prefer test before
implementation.

This is where I make my point.

As you work the problem and iterate over the implement-test cycle, you
spend time debugging.  The debugging process can be costly (how many
times is it one stinking semicolon, or the unexpected misuse of a
builtin function instead of your own that throws you for 4 hours?). 
That cost should be minimized, so that future developers do not share
in your plight.  How can you possibly pass on your knowledge w/out
representing it in test code?  You can't.

You've already chosen to make the investment in time to verify the
accuracy of your implementation.  If you're quick on the keyboard, it
only takes an extra 10% of overhead to formalize your tests in
automated regression format.  Why anyone would go 9 yards out of 10,
and then *not* represent their debugging work and testing in
regressable format is boggling.  Plus, then you get to hand over solid
proof that your module(s) behave according to design, which directly
relates to requirements.  This is your proof that you are a valuable
developer, and that you should be paid more (because you prove to your
managers that you're implementing customer requirements and that your
implementation works, which implies that your company will get
$$paid$$).

It boils down to this - visual inspection of spot test output is not a
reliable means by which to test software.  Neither is a poke and stab
approach where you flex one or two aspects of the requirements.  When
things go wrong (and they always do), a low level unit test approach
helps to eliminate ambiquity as to which modules work as designed, and
which are broken.

The real question (imo) is getting people to the level where they
recognize this view, support this view, and carry on this view so that
when you leave the project will continue to be successful.


-c


Edvard Majakari <edvard+web at majakari.net> wrote in message news:<87of0pno5s.fsf at titan.staselog.com>...
> I'm rather interested in (and fond of) unit testing as well as TDD, so I
> though to share my opinions. I once discussed TDD with my friend in here,
> and the common agreement seemed to be that TDD is not a silver bullet -
> often there are types of computational modules that are very hard to unit
> test, eg. think of graphical user interfaces or complex systems, where
> forking or threading is concerned. 
> 
> I tried to think of quick rule when to write unit tests for a module or
> not, and I only came up with this: if the whole module works like a
> set of functions (may be only one f, of course) where 
> 
> 1. Input is easily representable as native language data structures or
>    combination of those
> 
> 2. Output is easily representable as native language data structures or
>    combination of those
> 
> 3. Relationship between those two is deterministic
> 
> 
> Then I thought that if number of all (obvious) test cases is very large,
> it may not be worth it to write unit tests for all of them if the
> functions/methods concerned are simple enough.
> 
> For example, a software that reads XML and converts certain some XML
> snippets to Python data structures may contain very large number of
> combinations considering all possible valid XML files, but it may be still
> worth the trouble to write unit tests for at least most typical
> situations. You can always add more tests later.
> 
> I think one of the best things in writing unit tests is how it helps to
> avoid doing same mistakes twice. When you encounter a bug, you add
> appropriate test case to your test suite and fix it until all tests pass
> again. By running unit tests before every release you can be sure that all
> known bugs have been corrected (assuming every accepted bug results in new
> written test case).
> 
> Some people seem to have lukewarm attitude towards unit testing and
> TDD. No, TDD is not a silver bullet and neither unit testing nor TDD - or
> both combined - can guarantee a 100% bug-free software product, but it
> sure helps a lot and is a nice method in addition to many others in an
> attempt to design, implement and maintain high-quality software products.
> 
> Just my 2 cents,