[Python-Dev] Trimming the fat from "make quicktest" (was Re: I am now lost - committed, pulled, merged, what is "collapse"?)

[Nick Coghlan]

On Thu, Mar 24, 2011 at 7:07 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> On 23.03.2011 18:10, Barry Warsaw wrote:
>> On Mar 23, 2011, at 05:09 PM, Antoine Pitrou wrote:
>>
>>>That's completely bogus. There's no reason to believe that a push race would
>>>favour certain regressions over certain others. Again, you need the full test
>>>suite to assert that no regressions occured.  (or you might as well run 10
>>>tests at random and call it done)
>>
>> If you promote the full test suite as the thing to run when resolving merge
>> races, then I predict no one will run them, because doing so increases your
>> chances of hitting *another* push race.  This whole thread came up in the
>> context of trying to find a quick test you could run in that case which didn't
>> increase that race window.  I think the practical effect of not having a
>> simple, fast smoke test will be to do *less* testing when you hit the merge
>> race, and just let the buildbots sort it all out.  You'll probably win most of
>> the time anyway.
>
> FWIW, +1 to this.

To make it clear as to the use case Barry and I are trying to cover
here, when you get into a full push race for a bug fix, the current
work flow (in practice) is to just merge/commit/push. When you
multiply it by 3 branches, a useful smoke test needs to be *damn* fast
(i.e. less than a minute) because you're going to be running it three
times in quick succession (perhaps 3 if it applies to 2.7 as well).

The alternative is *not* a full test run, but at best simply running
the specific tests for whatever I'm fixing (or, more likely, not
running any tests at all).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia