[Python-Dev] My thinking about the development process

[Terry Reedy]

On 12/6/2014 10:26 AM, Nick Coghlan wrote:
> On 7 December 2014 at 01:07, Donald Stufft <donald at stufft.io> wrote:
>> A likely solution is to use a pre-merge test runner for the systems that we
>> can isolate which will give a decent indication if the tests are going to
>> pass across the entire supported matrix or not and then continue to use the
>> current post-merge test runner to handle testing the esoteric systems that
>> we can’t work into the pre-merge testing.
>
> Yep, that's exactly the approach I had in mind for this problem.

Most patches are tested on just one (major) system before being 
committed.  The buildbots confirm that there is no oddball failure 
elsewhere, and there is usually is not.  Testing user submissions on one 
system should usually be enough.

Committers should generally have an idea when wider testing is needed, 
and indeed it should be nice to be able to get wider testing on occasion 
*before* making a commit, without begging on the tracker.

What would be *REALLY* helpful for Idle development (and tkinter, 
turtle, and turtle demo testing) would be if there were a 
test.support.screenshot function that would take a screenshot and email 
to the tracker or developer.  There would also need to be at least one 
(stable) *nix test machine that actually runs tkinter code, and the 
ability to test on OSX with its different graphics options.  Properly 
testing Idle tkinter code that affects what users see is a real bottleneck.

-- 
Terry Jan Reedy