[Python-Dev] Buildbots and regrtest timeout
victor.stinner at haypocalc.com
Mon Jun 6 12:29:46 CEST 2011
Stephan Krah asked me to change how the default timeout is defined for regrtest
"The implicit timeout in regrtest.py makes it harder to write automated
test scripts for 3rd party modules. First, you have to remember to
set --timeout=0 for long running tests. Then, you have to remember
not to use the --timeout option when compiling --without-threads.
I'd much prefer that there's no timeout unless explicitly specified.
For the buildbots, I think this could be done in the Makefile."
First I replaced the hardcoded constant in regrtest.py by a command line
argument (--timeout=3600) in the TESTOPTS variable of the Makefile, so if you
call regrtest directly (without make), there is no more default timeout.
But today I saw a a buildbot timeout without any traceback: a possible hang in
test_io on "x86 FreeBSD 7.2 3.x" buildbot, "command timed out: 3900 seconds
without output". I realized that some buildbots (all buildbots?) override the
TESTOPTS variable ("make buildbottest TESTOPTS= TESTPYTHONOPTS=" for "x86
FreeBSD 7.2 3.x").
I moved the timeout option from TESTOPTS to a new variable: TESTTIMEOUT, with
a default value of 1 hour (3600 seconds). The timeout is now only used for
"make buildbottest". Use TESTTIMEOUT=0 to disable completly regrtest timeout.
The timeout argument if ignored (with a warning) if
faulthandler.dump_tracebacks_later() function is missing (e.g. if Python is
compiled without threads).
Please, don't disable the TESTTIMEOUT option for your buildbot. You may want
to use a shorter or longer timeout, it just have to be smaller than the
buildbot timeout (3900 seconds by default, 1 hour 6 minutes, just a little bit
more than regrtest timeout: 1 hour). If you want a timeout longer than 1 hour,
you can to change TESTTIMEOUT and the buildbot timeout (I don't know how to
configure the buildbot timeout).
I didn't touch test, testall, testuniversal, quicktest make rules, which don't
use the regrtest timeout anymore. I would prefer to use the same timeout
options for all test rules in TESTOPTS, but it doesn't work because some
buildbots do override TESTOPTS variable.
More information about the Python-Dev