Failing unittest Test cases

[Jean-Paul Calderone]

On 10 Jan 2006 13:49:17 -0800, Paul Rubin <"http://phr.cx"@nospam.invalid> wrote:
>skip at pobox.com writes:
>> Got any ideas how that is to be accomplished short of jiggering the
>> names so they sort in the order you want them to run?
>
>How about with a decorator instead of the testFuncName convention,
>i.e. instead of
>
>   def testJiggle():   # "test" in the func name means it's a test case
>      ...
>
>use:
>
>    @test
>    def jiggletest():  # nothing special about the name "jiggletest"
>       ...
>
>The hack of searching the module for functions with special names was
>always a big kludge and now that Python has decorators, that seems
>like a cleaner way to do it.

It's neither a hack nor a kludge.  It's introspection and it's a 
valuable technique for solving various problems. 

Just because something has been around for a long time does not 
necessarily mean it's bad.

>
>In the above example, the 'test' decorator would register the
>decorated function with the test framework, say by appending it to a
>list.  That would make it trivial to run them in code order.

Not that I think this is a bad idea, but ordering things based on 
the side-effects of a decorator seems rather magical.  This, on top 
of the fact that unit tests are only unit tests as long as they are 
only testing a something relatively isolated (let's call it a "unit") 
an, would make me wary of using this technique in any unit tests I 
write. 

The kind of thing being discussed is not without value - but it 
falls into the realm of integration testing, not unit testing. 
I'm not just slinging vocabulary around here, either.  It's 
important to keep the two kinds of tests separate.  Integration 
tests are great for telling you that something has gone wrong, 
but if you get rid of all your unit tests in the process of writing 
integration tests, you will have a more difficult time determining 
*what* has gone wrong.

Jean-Paul