[Python-Dev] New work-in-progress bisection tool for the Python test suite (in regrtest)

[Victor Stinner]

Another example to bisect "Running test_warnings twice fails" bug:
http://bugs.python.org/issue30812

---
haypo at selma$ ./python -m test.bisect -o bisect test_warnings  test_warnings
(...)
Tests (1):
* test.test_warnings._WarningsTests.test_showwarnmsg_missing
(...)
Bisection completed in 14 iterations and 0:00:14
---

So the bisection also works to find a failing test method when running
a test file twice ;-)

Victor

2017-06-28 2:39 GMT+02:00 Victor Stinner <victor.stinner at gmail.com>:
> 2017-06-27 7:33 GMT+02:00 Serhiy Storchaka <storchaka at gmail.com>:
>> You could make it just a submodule in the test package.
>>
>>     ./python -m test.bisect -R 3:3 test_os
>
> I like the idea :-) I proposed a PR which was approved by Yury
> Selivanov, and so I just merged it! It means that you can now play
> with "./python -m test.bisect" in the master branch. Enjoy ;-)
> Example:
>
>    ./python -m test.bisect -R 3:3 test_multiprocessing_forkserver
>
> This command should give you the name of the two failing test methods
> which are the last known reference leaks!
> => http://bugs.python.org/issue30775
>
> This specific bisection is very slow since running the 286 tests of
> test_multiprocessing_forkserver using -R 3:3 takes longer than 3
> minutes on my laptop! And the answer is already know, see the bpo ;-)
>
> FYI, apart of the bpo-30775, all other reference leaks now seem to be
> fixed on 2.7, 3.5, 3.6 and master branches! Tested on Windows and
> Linux.
>
> Victor