[Python-Dev] Spurious Buildbot Reports
titus at caltech.edu
Thu Dec 20 03:06:44 CET 2007
On Wed, Dec 19, 2007 at 05:58:35PM -0800, Brett Cannon wrote:
-> On Dec 19, 2007 4:33 PM, Raymond Hettinger <python at rcn.com> wrote:
-> > The bots are kicking-off so many false alarms that it is becoming difficult to tell whether a check-in genuinely broke a build.
-> > At the root of the problem is a number of tests in the test suite that randomly blow-up. I now tend to automatically dismiss failures in test_logging and test_threading for example.
-> Yeah, certain tests need some TLC to make them more predictable and
-> less prone to throw a failure because of some touch race condition or
-> something on the machine was not available to make the test work.
-> As I have stated on my blog, once I am done with importlib and the
-> stdlib reorg I plan on working on dev docs and then attack the whole
-> structure of the unit tests.
-> But who knows when that will happen. =)
OK, but this isn't really a structural problem, right? This is
non-determinism in certain tests ;)
How broken are these tests? Can we point a 17-yr-old at them and tell
them to fix 'em (yes think "GHOP")? (If the problem is reproducible on a
1-in-10 basis then I think the answer is "yes".)
And are test_threading and test_logging the two that need the most
Hmm, perhaps a good starting task would be to run the tests 10-100
times, in random order, on a single machine, to get a statistical
picture of of the problem.
(always on the lookout for core => GHOP tasks)
More information about the Python-Dev