[Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py

[Fredrik Lundh]

Neal Norwitz wrote:

> In the past, we haven't checked in tests which are known to be broken.
>  There are several good reasons for this.  I would prefer you, 1) also
> fix the code so the test doesn't fail, 2) revert the change (there's
> still a bug report open, right?), or 3) generalize tests for known
> bugs.
>
> I strongly prefer #1, but have been thinking about adding #3.  There
> are many open bug reports that fall into two broad categories:
> incorrect behaviour and crashers.  I've been thinking about adding two
> tests which incorporate these bugs as a way of consolidating where the
> known problems are.  Also, it's great when we have test cases that can
> be moved to the proper place once the fix has been checked in.

many test frameworks support "expected failures" for this purpose.
how hard would it be to add a

    unittest.FailingTestCase

class that runs a TestCase, catches any errors in it, and signals an
error ("test foo passed unexpectedly") if it runs cleanly ?

</F>