[Tutor] Unit testing infinite loops

[Steven D'Aprano]

On Fri, Jan 31, 2014 at 02:03:13PM +0000, James Chapman wrote:

> On the note of what doesn't need a test...
> The question of coverage always comes up when unit testing is
> mentioned and I read an interesting blog article once about it. It
> basically said: Assume you have 85% coverage on a program that
> consists of 1,000 lines. That's 150 lines which are not tested. If
> those lines are print lines, sleep lines, getters etc it's not really
> a problem. But what happens when you scale that up. 1,000,000 lines of
> code lets say (not unheard of, although in python that would be out of
> this world big). You now end up with 150,000 lines of untested code.
> While the percentage of code covered is high, there is _a_lot_ of code
> there that isn't tested and a lot of room for mistakes to creep in. A
> mistake on one of those 150,000 lines could break the build and
> possibly cost you hours or even days tracking it down. 

Or weeks, or months... 

You're right of course. But look at it this way. How much time, effort, 
money, and lost opportunities to be doing other things, are you willing 
to spend to asymptotically approach 100% coverage? It might take a week 
of development writing nothing but tests to get to 80% coverage, another 
week to get to 85%, a further week to get to 87%, another week again to 
get to 88%... 

At some point you say, "I have better things to do." Or your customers 
start questioning why the bills keep coming but the features aren't 
being delivered. (Customers want features, not tests.)

I'm not trying to talk you out of testing this. I agree completely with 
this:

> If those lines
> were tested however, your continuous integration build system would
> hopefully highlight the fault.
> 
> In my experience testing works, saves time down the line, and makes
> code easier to come back to.


but life is short, and even if you like writing tests, there comes a 
point of diminishing returns. And the ideal of programming is to write 
code which is obviously correct (as opposed to code which merely 
contains no obvious bugs).

And of course: better 85% coverage than 50%. Better 50% coverage 
than 10%. Better 10% coverage than no tests at all.


-- 
Steven