[Python-Dev] [ python-Bugs-1124637 ] test_subprocess is far too slow (fwd)

[Guido van Rossum]

> I'd like to have your opinion on this bug. Personally, I'd prefer to keep
> test_no_leaking as it is, but if you think otherwise...
> 
> One thing that actually can motivate that test_subprocess takes 20% of the
> overall time is that this test is a good generic Python stress test - this
> test might catch some other startup race condition, for example.

A suite of unit tests is a precious thing. We want to test as much as
we can, and as thoroughly as possible; but at the same time we want
the test to run reasonably fast. If the test takes too long, human
nature being what it is, this will actually cause less thorough
testing because developers don't feel like running the test suite
after each small change, and then we get frequent problems where
someone breaks the build because they couldn't wait to run the unit
test.

(For example, where I work we have a Java test suite that takes 25
minutes to run. The build is broken on a daily basis by developers
(including me) who make a small change and check it in believing it
won't break anything.)

The Python test suite already has a way (the -u flag) to distinguish
between "regular" broad-coverage testing and deep coverage for
specific (or all) areas. Let's keep the really long-running tests out
of the regular test suite.

There used to be a farm of machines that did nothing but run the test
suite ("snake-farm"). This seems to have stopped (it was run by
volunteers at a Swedish university). Maybe we should revive such an
effort, and make sure it runs with -u all.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)