Art of Unit Testing

[phil hunt]

On Wed, 03 Aug 2005 09:51:49 -0400, Peter Hansen <peter at engcorp.com> wrote:
>phil hunt wrote:
>> On Tue, 02 Aug 2005 21:26:28 +0200, Christoph Zwerschke <cito at online.de> wrote:
>>>According to the "extreme programming" paradigm, testing should be done 
>>>several times a day. So a requirement for extreme programm is that tests 
>>>are fast enough. If the testing needs too much time, people are 
>>>discouraged to test often.
>> 
>> Indeed. Running the tests should ideally take less than a few 
>> seconds. Any longer, and people won't use them so often.
>
>That's probably too general a statement to be very useful, and I don't 
>think it's probably helpful to a newcomer who is seeking guidance.

I did say "ideally"; if the test have to take longer, so be it.

>If one makes the XP-style distinction between "unit" and "acceptance" 
>tests, then the acceptance test *suite* should take no more than some 
>number of minutes.  I believe ten minutes absolute max is a commonly 
>offered rule of thumb, but I've found even eight minutes a little too 
>long.  Five seems to be just fine, and less if of course wonderful.

I think we might be talking at cross purposes here. To me 
"acceptance test suite" means a test suite that has to be passed 
each time before a new version of the software is released to the 
users. I don't see that 10 minutes is a sensible limit here, unless 
you are releasing more often that once a day. (I once had an 
acceptance test suite that ran for 12 hours; I used to run it 
nightly).

>If you're doing 
>test-driven development (TDD), you will want to be able to run unit 
>tests (though maybe not all of them) sometimes a couple of times per 
>minute! 

That's what I often do.

>Still, in a large project (and especially one written in Python, with 
>the overhead of interpreter startup and the cost of executing bytecode) 
>the suite can get fairly long if you have many hundreds of tests.

I'm currently running one with 227 assertions in 38 test functions.
It runs in <1 second on an AMD 3000+.


-- 
Email: zen19725 at zen dot co dot uk