[Python-Dev] Allowing to run certain regression tests in subprocesses

[Eli Bendersky]

Hi All,

Today the issue of cross-test global env dependencies showed its ugly
head again for me. I recall a previous discussion
(http://mail.python.org/pipermail/python-dev/2013-January/123409.html)
but there were many more over the years.

The core problem is that some tests modify the global env
(particularly importing modules) and this sometimes has adverse
effects on other tests, because test.regrtest runs all tests in a
single process. In the discussion linked above, the particular culprit
test__all__ was judged as a candidate to be moved to a subprocess.

I want to propose adding a capability to our test harness to run
specific tests in subprocesses. Each test will have some simple way of
asking to be run in a subprocess, and regrtest will concur (even when
running -j1). test__all__ can go there, and it can help solve other
problems.

My particular case is trying to write a test for
http://bugs.python.org/issue14988 - wherein I have to simulate a
situation of non-existent pyexpat. It's not hard to write a test for
it, but when run in tandem with other tests (where C extensions loaded
pyexpat) it becomes seemingly impossible to set up. This should not be
the case - there's nothing wrong with wanting to simulate this case,
and there's nothing wrong in Python and the stdlib - it's purely an
artifact of the way our regression suite works.

Thoughts?

Eli