[New-bugs-announce] [issue38865] Can Py_Finalize() be called if the current interpreter is not the main interpreter?

[STINNER Victor]

New submission from STINNER Victor <vstinner at python.org>:

Programs/_testembed.c contains the following test used by test_embed:

static int test_audit_subinterpreter(void)
{
    Py_IgnoreEnvironmentFlag = 0;
    PySys_AddAuditHook(_audit_subinterpreter_hook, NULL);
    _testembed_Py_Initialize();

    Py_NewInterpreter(); 
    Py_NewInterpreter(); 
    Py_NewInterpreter(); 
 
    Py_Finalize(); 

    switch (_audit_subinterpreter_interpreter_count) {
        case 3: return 0;
        case 0: return -1;
        default: return _audit_subinterpreter_interpreter_count;
    }
}

When Py_Finalize() is called, the current interpreter is a subinterpreter (the 3rd interpreter), not the main interpreter.

* Is it correct to call Py_Finalize() in such case?
* Is Python supposed to magically destroy the 3 interpreters?

In bpo-38858, I'm trying to reuse the same code to initialize and finalize the "main" interpreter and subinterpreters. I had an issue with test_audit_subinterpreter() when working on the PR 17293.

I modified my PR 17293 to not expect that Py_Finalize() can only be called from the main interpreter, but actually check if the current interpreter is the main interpreter or not. It fix test_audit_subinterpreter() but again, I'm not sure what is the correct behavior?

--

Last year, we had a similar discussion about calling Py_Main() *after* Py_Initialize(). I hacked the code to make it possible because it was supported previously, even if the Py_Main() configuration is only partially applied. But I understood that Nick Coghlan would prefer to deprecate supporting to call Py_Initialize() before Py_Main().

PEP 587 added Py_RunMain() which provides a different solution to this problem:
https://docs.python.org/dev/c-api/init_config.html#c.Py_RunMain

----------
components: Interpreter Core
messages: 357080
nosy: eric.snow, nanjekyejoannah, ncoghlan, vstinner
priority: normal
severity: normal
status: open
title: Can Py_Finalize() be called if the current interpreter is not the main interpreter?
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38865>
_______________________________________