[Python-ideas] unittest and warnings

[Barry Warsaw]

On Jun 17, 2013, at 08:22 AM, Masklinn wrote:

>
>On 2013-06-17, at 02:32 , Giampaolo Rodola' wrote:
>
>> One of the features I like the most in the new unittest2 module is the
>> possibility to skip tests and how they are included in the final result
>> (e.g. FAILED (errors=2, failures=3, *skipped=5*)).
>> After http://bugs.python.org/issue10093 it is not rare that different
>> ResourceWarnings appear while running tests.
>> Personally I consider these warnings as something which needs to be fixed
>> so after I run tests I often scroll my console window back up to see
>> whether there were warnings.
>> Would it make sense for unittest module to keep track of them so that they
>> get included in the final test result as it currently happens for skipped
>> tests?
>> 
>> Side note: unittest provides some "skip-related" APIs such unittest.skip*,
>> TestResult.skipped and others but I don't think something similar would be
>> necessary except maybe a TestResult.warnings list similar to
>> TestResult.skipped.
>
>Why not just use -Werror?

That doesn't really work so well.  Those errors don't get turned into
exceptions that cause tests to fail, so they still don't show up in the test
suite's summary.  ResourceWarnings are particularly difficult to track down,
as described in my previous message to this list.

+1 for Giampaolo's original suggestion that skipped tests be described in the
summary.  I also have to scroll back to find the ResourceWarnings, and I find
that additional pain in trying to debug these things.

>Alternatively, capture warnings[0] and report them as whatever you wish
>to your test system.

Yeah, not so much.  They can be quite unpredictable, so there's no good place
to set up the captures.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20130617/c87fb33d/attachment.pgp>