[Tutor] the art of testing
Nick
reply_to_is_valid at nowhere.com.invalid
Wed Nov 25 04:41:29 CET 2009
As well as the other replies, consider that you are doing "unit" testing:
http://en.wikipedia.org/wiki/Unit_test
One method is black-box testing, which is where the thing (class,
function, module) you are testing is treated as a black box, something
that takes input and returns output, and how it does it are irrelevant
from the tester's perspective. You remove the implementation of the thing
from the test of the thing, which allows you to focus on the tests. In
fact, you can write the tests before you code up the black box, which is a
valid development technique that has many proponents.
You throw all sorts of data at the black box, particularly invalid,
boundary and corner-case data, and test that the black box returns
consistent, valid output in all cases, and that it never crashes or
triggers an exception (unless part of the design).
Once you have run your tests and have produced a set of test failures, you
have some leads on what within the black box is broken, and you take off
your tester's hat, put on your developer hat, and go fix them. Repeat. A
pre-built test suite makes this easier and gives consistency. You know
that your input data is the same as it was before your changes and you can
test for consistency of output over your development cycle so that you
know you haven't inadvertently introduced new bugs. This process is
regression testing.
Clearly, ensuring your test data covers all possible cases is important.
This is one reason for designing the test suite before building the black
box. You won't have inadvertently tainted your mind-set with preconceived
notions on what the data contains, since you haven't examined it yet.
(You've only examined the specs to find out what it *should* contain; your
black box implementation will handle everything else gracefully, returning
output when it should and triggering an exception when it should.) This
frees you up to create all sorts of invalid, i.e. non-specification, and
boundary test data, without preconceived ideas.
Once you are passing your test data, throw lots of samples of real-life
data at it. If your test data was comprehensive, real-life data should be
a piece of cake.
Obviously I'm a fan of unit-testing. Sure, the first time they're a bit of
work to build up, but you'll find you can re-use them over and over with a
small amount of editing. Many conditions are the same for any program,
such as (off the top of my head) file-not-found, file-has-one-record-only,
file-has-a-trillion-records, string-where-int-expected, int-out-of-
expected-range, and so on.
More information about the Tutor
mailing list