[Python-Dev] Allowing to run certain regression tests in subprocesses

[Eli Bendersky]

On Sat, Aug 3, 2013 at 6:59 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
> On 4 Aug 2013 09:43, "Eli Bendersky" <eliben at gmail.com> wrote:
>>
>> Hi All,
>>
>> Today the issue of cross-test global env dependencies showed its ugly
>> head again for me. I recall a previous discussion
>> (http://mail.python.org/pipermail/python-dev/2013-January/123409.html)
>> but there were many more over the years.
>>
>> The core problem is that some tests modify the global env
>> (particularly importing modules) and this sometimes has adverse
>> effects on other tests, because test.regrtest runs all tests in a
>> single process. In the discussion linked above, the particular culprit
>> test__all__ was judged as a candidate to be moved to a subprocess.
>>
>> I want to propose adding a capability to our test harness to run
>> specific tests in subprocesses. Each test will have some simple way of
>> asking to be run in a subprocess, and regrtest will concur (even when
>> running -j1). test__all__ can go there, and it can help solve other
>> problems.
>>
>> My particular case is trying to write a test for
>> http://bugs.python.org/issue14988 - wherein I have to simulate a
>> situation of non-existent pyexpat. It's not hard to write a test for
>> it, but when run in tandem with other tests (where C extensions loaded
>> pyexpat) it becomes seemingly impossible to set up. This should not be
>> the case - there's nothing wrong with wanting to simulate this case,
>> and there's nothing wrong in Python and the stdlib - it's purely an
>> artifact of the way our regression suite works.
>
> I'm not actively opposed to the suggested idea, but is there a specific
> reason "test.support.import_fresh_module" doesn't work for this test?

I'm not an expert on this topic, but I believe there's a problem
unloading code that was loaded by C extensions. import_fresh_module is
thus powerless here (which also appears to be the case empirically).

Eli