[New-bugs-announce] [issue12400] regrtest: always run tests in verbose mode, but hide the output on success
report at bugs.python.org
Fri Jun 24 22:45:59 CEST 2011
New submission from STINNER Victor <victor.stinner at haypocalc.com>:
Sporadic issues are difficult to analyze on the buildbots because sometimes a test fails, but when it runs again in verbose mode... it doesn't fail anymore. Typical example:
[ 99/356/1] test_smtplib
Re-running test test_smtplib in verbose mode
testTimeoutValue (test.test_smtplib.GeneralTests) ... ok
testBasic (test.test_smtplib.DebuggingServerTests) ... test test_smtplib failed -- multiple errors occurred; run in verbose mode for details
testHELP (test.test_smtplib.DebuggingServerTests) ... ok
testNOOP (test.test_smtplib.DebuggingServerTests) ... ok
testNotImplemented (test.test_smtplib.DebuggingServerTests) ... ok
Ran 32 tests in 23.664s
I propose to always run the tests in verbose mode, write the output into a buffer, but only display the output on failure (or ignore the output on success).
Attached patch simplifies regrtest.py usage: replace 4 verbose levels (no verbose option, -v, -w, -W) by only 2 (no verbose option, -v). Always display the full output in case of an error (as we used -w before): no more "multiple error occured" without output trap, no more "oh, rerunning the test doesn't fail" trap. The -v option works as before (directly write the output in verbose mode).
regrtest is much more verbose than before in case of a failure, but I prefer too much information than no information when I try to debug something. If it is too much verbose by default, I can try to use 3 verbose levels (e.g. don't display the full output if only one test failed).
If the patch is accepted, the devguide should also be patched.
nosy: ezio.melotti, georg.brandl, haypo, pitrou, r.david.murray
title: regrtest: always run tests in verbose mode, but hide the output on success
versions: Python 2.7, Python 3.2, Python 3.3
Added file: http://bugs.python.org/file22446/regrtest_verbose.patch
Python tracker <report at bugs.python.org>
More information about the New-bugs-announce