[Python-Dev] I am now lost - committed, pulled, merged, what is "collapse"?

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 3/22/2011 11:05 PM, Barry Warsaw wrote:
> On Mar 23, 2011, at 07:31 AM, Nick Coghlan wrote:
> 
>> On Wed, Mar 23, 2011 at 4:25 AM, Barry Warsaw <barry at python.org> wrote:
>>> It probably wouldn't be a bad idea to add a very fast "smoke test" for the
>>> case where you get tripped up on the merge dance floor.  When that happens,
>>> you could run your localized tests, and then a set of tests that run in just a
>>> minute or two.
>>
>> What would such a smoke test cover, though? It's hard to think of
>> anything particularly useful in the middle ground between "Can you run
>> regrtest *at all*?" and "make quicktest".
> 
> make quicktest runs 340 tests, and I'm certain many don't need to be run in a
> smoke test.  E.g. 
> 
> test_aifc
> test_colorsys
> test_concurrent_futures
> (that's as far as it's gotten so far ;)
> 
> Or you could time each individual test, choose a cut off for total test run
> and then run whatever you can within that time (on some reference machine).
> Or maybe just remove the longest running 50% of the quick tests.
> 
> -Barry

bzr can run its 30,000 tests in about 15-30min depending on your
specific hardware and platform (it also supports parallel running of the
test suite). I think it takes about 1hr running single-threaded on
Windows because "os.rename()" is a fairly slow operation and we do it a
lot in most tests (it is part of our locking primitive).


Unit-tests (tests directly on a function without setting up lots of
external state) generally can execute in milliseconds.

I don't specifically know what is in those 340 tests, but 18min/340 =
3.2s for each test. Which is *much* longer than simple smoke tests would
have to be.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2J3WUACgkQJdeBCYSNAAMI2ACgl6obH9WKlmRiK4K4ib1g6SR7
KqkAn1LNrlBaUTf/sc5s30tZq3F3hmNl
=DtQ8
-----END PGP SIGNATURE-----