[Python-Dev] Buildbots now always run tests in subprocesses

[Victor Stinner]

Hi,

I modified regrtest to not ignore -j1 anymore (issue #25285). I also
modified Tools/buildbot/test.bat to add the -j1 option (-j1 was
already used on UNIX on "make buildbottest").

All buildbots will now run each test file in a fresh subprocess. It
helps to restrict all kinds of side effects of unit tests to a single
file. The drawback is that buildbots will be less efficient to detect
side effects of unit tests.


By the way, I also changed regrtest output in "multiprocess mode"
(when -jN is used) to add "passed" to test names:
---
$ ./python -m test -j1
...
Run tests in parallel using 1 child processes
0:00:00 [  1/401] test_grammar passed
0:00:00 [  2/401] test_opcodes passed
0:00:01 [  3/401] test_dict passed
---

When tests are run sequentially, the output is different:
---
$ ./python -u -m test
...
Run tests sequentially
0:00:00 [  1/401] test_grammar
0:00:00 [  2/401] test_opcodes
0:00:00 [  3/401] test_dict
...
---

In multiprocess mode, regrtest displays the name of the test which
just completed. In sequential mode, it displays the name of the test
which is currently running.

Maybe the output of sequential mode can also be enhanced, but I made
the minimum change to avoid breaking regrtest. Some parts of the code
rely on the regrtest output format. I added many unit tests to
test_regrtest, but not everything is tested yet.

Victor