[Tutor] another unit-testing question: regex testing

[Kent Johnson]

On Tue, Dec 15, 2009 at 5:56 PM, Serdar Tumgoren <zstumgoren at gmail.com> wrote:
> Hi everyone,
>
> To continue on the popular topic of unit tests, I was wondering if
> it's generally preferable to use a single unit test or a series of
> unit tests when testing against a regex pattern.
>
> In my specific case, I started out with a single test that contained a
> (growing) list of bad input data intended to fail when tried against a
> regex pattern.
>
> But then I started thinking: Isn't each unit test supposed to test one
> thing and one thing only? So I broke up my list of inputs and devoted
> a single unit test to each (see below for both of my iterations).
>
> On the one hand, I can see the benefit of having individual unit tests
> for each of these varying inputs, since I can then see precisely which
> one failed when I run my tests.
>
> On the other hand, this one-test-per-pattern seems like a bit of
> overkill given that I built up this list of bad inputs sequentially.
> In other words, I added the first bad input to my initial list (in
> "testIDFormat"), then ran the tests; then I added another failing
> input and ran the tests; and so on.
>
> In that way, I figured I could gradually build up a list of known bad
> inputs, and simply append any new bad inputs that crop up later on.
>
> So my question -- in this particular case involving a regex, what
> downsides are there of simply lumping all the known bad values into a
> single unit test?

I often use a list of test cases to drive a single test. Using a
series of tests is just too painful compared to making a simple list
of test cases.

The disadvantages to using a list of test cases:
- it will die at the first failure, which may hide other failures
- the default failure messages are generally not helpful - they won't
necessarily tell you which test case failed. I usually build a custom
failure message and use the assertXxx() method that takes a message
parameter. (In your case, you will have to use a try / catch with a
fail() in the try block because assertRaises() doesn't take a msg
parameter.)

Some of the alternate test runners (nose, py.test) may have better
support for this style of testing but it works OK in unittest.

Kent