[Tutor] the art of testing

Nick reply_to_is_valid at nowhere.com.invalid
Wed Nov 25 04:41:29 CET 2009


As well as the other replies, consider that you are doing "unit" testing:
http://en.wikipedia.org/wiki/Unit_test

One method is black-box testing, which is where the thing (class, 
function, module) you are testing is treated as a black box, something 
that takes input and returns output, and how it does it are irrelevant 
from the tester's perspective.  You remove the implementation of the thing 
from the test of the thing, which allows you to focus on the tests.  In 
fact, you can write the tests before you code up the black box, which is a 
valid development technique that has many proponents.

You throw all sorts of data at the black box, particularly invalid, 
boundary and corner-case data, and test that the black box returns 
consistent, valid output in all cases, and that it never crashes or 
triggers an exception (unless part of the design).

Once you have run your tests and have produced a set of test failures, you 
have some leads on what within the black box is broken, and you take off 
your tester's hat, put on your developer hat, and go fix them.  Repeat.  A 
pre-built test suite makes this easier and gives consistency.  You know 
that your input data is the same as it was before your changes and you can 
test for consistency of output over your development cycle so that you 
know you haven't inadvertently introduced new bugs.  This process is 
regression testing.  

Clearly, ensuring your test data covers all possible cases is important.  
This is one reason for designing the test suite before building the black 
box.  You won't have inadvertently tainted your mind-set with preconceived 
notions on what the data contains, since you haven't examined it yet.  
(You've only examined the specs to find out what it *should* contain; your 
black box implementation will handle everything else gracefully, returning 
output when it should and triggering an exception when it should.)  This 
frees you up to create all sorts of invalid, i.e. non-specification, and 
boundary test data, without preconceived ideas.

Once you are passing your test data, throw lots of samples of real-life 
data at it.  If your test data was comprehensive, real-life data should be 
a piece of cake.

Obviously I'm a fan of unit-testing. Sure, the first time they're a bit of 
work to build up, but you'll find you can re-use them over and over with a 
small amount of editing.  Many conditions are the same for any program, 
such as (off the top of my head) file-not-found, file-has-one-record-only, 
file-has-a-trillion-records, string-where-int-expected, int-out-of-
expected-range, and so on.



More information about the Tutor mailing list