unittest: assertRaises() with an instance instead of a type

[Steven D'Aprano]

On Thu, 29 Mar 2012 08:35:16 -0700, Ethan Furman wrote:

> Steven D'Aprano wrote:
>> On Wed, 28 Mar 2012 14:28:08 +0200, Ulrich Eckhardt wrote:
>> 
>>> Hi!
>>>
>>> I'm currently writing some tests for the error handling of some code.
>>> In this scenario, I must make sure that both the correct exception is
>>> raised and that the contained error code is correct:
>>>
>>>
>>>    try:
>>>        foo()
>>>        self.fail('exception not raised')
>>>    catch MyException as e:
>>>        self.assertEqual(e.errorcode, SOME_FOO_ERROR)
>>>    catch Exception:
>>>        self.fail('unexpected exception raised')
>> 
>> Secondly, that is not the right way to do this unit test. You are
>> testing two distinct things, so you should write it as two separate
>> tests:
> 
> I have to disagree -- I do not see the advantage of writing a second
> test that *will* fail if the first test fails as opposed to bundling
> both tests together, and having one failure.

Using that reasoning, your test suite should contain *one* ginormous test 
containing everything:

def testDoesMyApplicationWorkPerfectly(self):
    # TEST ALL THE THINGS!!!
    ...


since *any* failure in any part will cause cascading failures in every 
other part of the software which relies on that part. If you have a tree 
of dependencies, a failure in the root of the tree will cause everything 
to fail, and so by your reasoning, everything should be in a single test.

I do not agree with that reasoning, even when the tree consists of two 
items: an exception and an exception attribute.

The problem of cascading test failures is a real one. But I don't believe 
that the solution is to combine multiple conceptual tests into a single 
test. In this case, the code being tested covers two different concepts:

1. foo() will raise MyException. Hence one test for this.

2. When foo() raises MyException, the exception instance will include an
   errorcode attribute with a certain value. This is conceptually 
   separate from #1 above, even though it depends on it. 

Why is it conceptually separate? Because there may be cases where the 
caller cares about foo() raising MyException, but doesn't care about the 
errorcode. Hence errorcode is dependent but separate, and hence a 
separate test.


-- 
Steven