[Python-Dev] Trimming the fat from "make quicktest" (was Re: I am now lost - committed, pulled, merged, what is "collapse"?)

[Nick Coghlan]

On Wed, Mar 23, 2011 at 11:31 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Wed, 23 Mar 2011 22:49:39 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
>> On Wed, Mar 23, 2011 at 9:45 PM, John Arbash Meinel
>> <john at arbash-meinel.com> wrote:
>> > I don't specifically know what is in those 340 tests, but 18min/340 =
>> > 3.2s for each test. Which is *much* longer than simple smoke tests would
>> > have to be.
>>
>> The counts Barry is referring to there are actually counting test
>> *files*, rather than individual tests. We only have 359 of those in
>> total though (not counting those in subdirectories), so a "quicktest"
>> that omits less than 6% of them doesn't sound particularly quick (even
>> if it does leave out the slowest ones).
>>
>> We should probably do another pass and add a few more tests to the
>> blacklist in the Makefile template (starting with
>> test_concurrent_futures).
>
> Does anyone use "make quicktest" for something useful?
> There is a reason the regression test suite has many tests...
> "Blacklisting" some of them sounds like a bad thing to do.

Oops, lost a bit too much context when I changed the thread title.

This discussion started with Barry looking for a "smoke test" that
would be quick enough to run that more people would be willing to use
it to pick up gratuitous breakage due to a bad merge rather than
leaving it for the buildbots to discover.

Currently even "make quicktest" takes too long to run to be suitable
for that task. Leaving out a couple more egregiously slow tests and
possibly updating it to use the "-j" switch might make for a usable
option.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia