[New-bugs-announce] [issue40231] Fix pending calls in subinterpreters

[STINNER Victor]

New submission from STINNER Victor <vstinner at python.org>:

Currently, _Py_ThreadCanHandlePendingCalls() only returns true if the current thread is the Python "main thread" (_PyRuntime.main_thread).

_PyRuntime.main_thread is initialized by _PyRuntime_Initialize().

The problem is that a subinterpreter can run a separated thread which may not be the "main thread".

As a consequence, a subinterpreter will not run schedulded pending calls, or it will run them later than it could.

I modified COMPUTE_EVAL_BREAKER() of ceval.c in bpo-40010: now if tstate->interp->ceval.pending.calls_to_do is true, tstate->interp->ceval.eval_breaker is only set to 1 if _Py_ThreadCanHandlePendingCalls() is true.

One option would be to allow any thread to run "pending calls".

Another option is to have one "main thread" per interpreter, rather than having a single "main thread" for all interpreters.


I made pending calls per-interpreter in bpo-39984.


In Python 3.7, main_thread variable came from _PyRutimeState.ceval.pending.main_thread. It was moved into _PyRuntimeState by this change:

commit 5be45a6105d656c551adeee7770afdc3b806fbb5
Author: Eric Snow <ericsnowcurrently at gmail.com>
Date:   Fri Mar 8 22:47:07 2019 -0700

    bpo-33608: Minor cleanup related to pending calls. (gh-12247)

--

_Py_ThreadCanHandleSignals() doesn't have to change: it must only return true for the main thread of the main interpreter. Currently, it's implemented as:

static inline int
_Py_ThreadCanHandleSignals(PyThreadState *tstate)
{
    return (_Py_IsMainThread() && _Py_IsMainInterpreter(tstate));
}

----------
components: Interpreter Core
messages: 366010
nosy: eric.snow, vstinner
priority: normal
severity: normal
status: open
title: Fix pending calls in subinterpreters
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue40231>
_______________________________________