On Wed, 23 Mar 2011 13:58:31 -0400 Tres Seaver <tseaver@palladion.com> wrote:
That doesn't work so well at a sprint, where the point is to maximize the value of precious face-time to get stuff done *now*. Long test latencies and nearly-real-time collaboration are not friendly, as the agile folks document:
http://c2.com/cgi/wiki?TestSpeed
Maybe we need to chop the problem up as:
- - Pure documentation changes never require running any test suite (this includes true comments in code, as well as docstrings which are not used to drive doctests or other tested output).
- - "core" language changes always require running the full test suite.
- - We compute an import-dependency map for the stdlib (maybe during build?), and add support for running tests of a named module and its dependents. Any non-documentation change to a stdlib module requires running this new kind of test against that module.
I agree that finding ways to speedup running tests *without* reducing the necessary coverage is the right way to go. Part of that is obviously about optimizing the tests themselves (something which I and others have been doing repeatedly, including the addition of the "-j" switch). The dependency map is another idea. All this needs work and is therefore more difficult than blacklisting some "slow" tests, but is much more productive in the long run. Regards Antoine.