[New-bugs-announce] [issue25687] Error during test case and tearDown
report at bugs.python.org
Sat Nov 21 02:54:27 EST 2015
New submission from Benno Leslie:
I'm not sure if this should be considered a bug, but the behaviour is surprising and not clearly documented.
I a have a very simple test that has an error during both the test case, and during tearDown.
if __name__ == '__main__':
When this occurs it is surprising (to me) that the output is:
Ran 1 test in 0.000s
In particular, the fact that has more errors than there are tests that have been run. Obviously in this very simple example it is clear what has happened, however in a test suite that has hundreds of test cases it is somewhat frustrating to have the number of failing test cases over-reported. (And of course in the real-world test suite that led to this the tearDown doesn't fail on every single test case like in this simplified example).
Although there are definitely two errors occurring in my example, in all other cases, only the first error would be reported. e.g.: an error in setUp wouldn't run the test case, and only the first (potential) error in the testcase itself would occur.
I think that either:
1/ The documentation of the tearDown method should clearly indicate that an error in tearDown would cause multiple errors to be reported for the single testCase, or.
2/ Change the implementation so that if there is an exception during tearDown, and there was already an error in the test case only the first error is reported.
2 is probably a non-starter given backwards compatibility concerns (even assuming anyone else thinks this is the best behaviour in this corner case).
I'd be happy to try and draft something for #1 if that is the best action.
components: Library (Lib)
title: Error during test case and tearDown
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4
Python tracker <report at bugs.python.org>
More information about the New-bugs-announce