[Python-Dev] Doctests REPORT_ONLY_FIRST_FAILURE ignored

[Lennart Regebro]

If you set up any sort of report flag on the unit test itself, the
default report flags given to the testrunner are ignored. This goes
for all report flags, the REPORT_xDIFF and REPORT_ONLY_FIRST_FAILURE.
I'd suggest that we do allow the testrunner to set
REPORT_ONLY_FIRST_FAILURE by default even if the testcase has a
REPORT_xDIFF flag. Why? Because it makes sense. :)

The REPORT_xDFF flags makes sense both as parameters to a testrunner,
and as flags on a testcase. You might want to permanently set diff
flags on tests that generate output that warrants a diff if they fail.

REPORT_ONLY_FIRST_FAILURE you would rarely set on a testcase. You
don't want that on the testcase, as buildbots wouldn't see the
subsequent fails, and developers might think it was only a minor
issue. It is a flag you give to the testrunner to stop having the
first failure scroll off screen. So you want it to work *always*. You
don't want or expect it to stop working just because one testcase had
a DIFF flag set. Right?

Or did I miss something? If not, I'll provide a patch and put it in
the bugtracker.

-- 
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64