[Python-Dev] Doctests REPORT_ONLY_FIRST_FAILURE ignored

[Lennart Regebro]

On Wed, Apr 21, 2010 at 06:56, Lennart Regebro <regebro at gmail.com> wrote:
> If you set up any sort of report flag on the unit test itself, the
> default report flags given to the testrunner are ignored. This goes
> for all report flags, the REPORT_xDIFF and REPORT_ONLY_FIRST_FAILURE.
> I'd suggest that we do allow the testrunner to set
> REPORT_ONLY_FIRST_FAILURE by default even if the testcase has a
> REPORT_xDIFF flag. Why? Because it makes sense. :)
>
> The REPORT_xDFF flags makes sense both as parameters to a testrunner,
> and as flags on a testcase. You might want to permanently set diff
> flags on tests that generate output that warrants a diff if they fail.
>
> REPORT_ONLY_FIRST_FAILURE you would rarely set on a testcase. You
> don't want that on the testcase, as buildbots wouldn't see the
> subsequent fails, and developers might think it was only a minor
> issue. It is a flag you give to the testrunner to stop having the
> first failure scroll off screen. So you want it to work *always*. You
> don't want or expect it to stop working just because one testcase had
> a DIFF flag set. Right?
>
> Or did I miss something? If not, I'll provide a patch and put it in
> the bugtracker.

I apparently didn't miss anything. :) Patch will arrive soon. :)