[Python-Dev] New work-in-progress bisection tool for the Python test suite (in regrtest)

Terry Reedy tjreedy at udel.edu
Wed Jun 28 00:52:55 EDT 2017

On 6/27/2017 8:39 PM, Victor Stinner wrote:
> 2017-06-27 7:33 GMT+02:00 Serhiy Storchaka <storchaka at gmail.com>:
>> You could make it just a submodule in the test package.
>>      ./python -m test.bisect -R 3:3 test_os
> I like the idea :-) I proposed a PR which was approved by Yury
> Selivanov, and so I just merged it! It means that you can now play
> with "./python -m test.bisect" in the master branch. Enjoy ;-)
> Example:
>     ./python -m test.bisect -R 3:3 test_multiprocessing_forkserver
> This command should give you the name of the two failing test methods
> which are the last known reference leaks!
> => http://bugs.python.org/issue30775
> This specific bisection is very slow since running the 286 tests of
> test_multiprocessing_forkserver using -R 3:3 takes longer than 3
> minutes on my laptop! And the answer is already know, see the bpo ;-)
> FYI, apart of the bpo-30775, all other reference leaks now seem to be
> fixed on 2.7, 3.5, 3.6 and master branches! Tested on Windows and
> Linux.

Are you testing for refleaks with gui enabled?  A few weeks ago I 
discovered that some had crept in IDLE, but the gui-less leak tests 
never saw them.  I fixed the ones that existed then, but as I add tests, 
it will be easy enough to add more leaks.

Can test.bisect run a test module in idlelib.idle_test?  Unittest.main 
will, but as far as I know, it does not run leak tests.  Test.regrtest 
has a leak test, but will not modules in other directories, at least not 
that one.  I wrote a custom script to narrow a leak in test.test_idle to 
one or more files in idlelib.idle_test, but from then on it was manual 
work, commenting out parts of the file until I identified the test 
function or functions.

One of the things I had to do is test setUpClass and tearDownClass by 
themselves by commenting out all test_xyz functions and inserting 
test_dummy(self): pass.

Terry Jan Reedy

More information about the Python-Dev mailing list