[New-bugs-announce] [issue25687] Error during test case and tearDown

[Benno Leslie]

New submission from Benno Leslie:

I'm not sure if this should be considered a bug, but the behaviour is surprising and not clearly documented.

I a have a very simple test that has an error during both the test case, and during tearDown.

"""
import unittest

class Test(unittest.TestCase):

    def test_a(self):
        asdf

    def tearDown(self):
        asdf

if __name__ == '__main__':
    unittest.main()
"""

When this occurs it is surprising (to me) that the output is:

"""
Ran 1 test in 0.000s

FAILED (errors=2)
"""

In particular, the fact that has more errors than there are tests that have been run. Obviously in this very simple example it is clear what has happened, however in a test suite that has hundreds of test cases it is somewhat frustrating to have the number of failing test cases over-reported. (And of course in the real-world test suite that led to this the tearDown doesn't fail on every single test case like in this simplified example).

Although there are definitely two errors occurring in my example, in all other cases, only the first error would be reported. e.g.: an error in setUp wouldn't run the test case, and only the first (potential) error in the testcase itself would occur.

I think that either:

1/ The documentation of the tearDown method should clearly indicate that an error in tearDown would cause multiple errors to be reported for the single testCase, or.
2/ Change the implementation so that if there is an exception during tearDown, and there was already an error in the test case only the first error is reported.

2 is probably a non-starter given backwards compatibility concerns (even assuming anyone else thinks this is the best behaviour in this corner case).

I'd be happy to try and draft something for #1 if that is the best action.

----------
components: Library (Lib)
messages: 255045
nosy: bennoleslie
priority: normal
severity: normal
status: open
title: Error during test case and tearDown
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25687>
_______________________________________