On Tue, Nov 12, 2019 at 3:11 PM Victor Stinner <vstinner@python.org> wrote:
Are you ok to modify internal C functions to pass explicitly tstate?
I'm also in favor (strongly)! (no surprises there) The only concerns I've heard is that on some platforms there is a measurable overhead once you hit a threshold of a specific small number of parameters. Adding this extra parameter will put some functions over that threshold. I don't have any more information than that.
There are different reasons to pass explicitly tstate, but my main motivation is to rework Python code base to move away from implicit global states to states passed explicitly, to implement the PEP 554 "Multiple Interpreters in the Stdlib". In short, the final goal is to run multiple isolated Python interpreters in the same process: run pure Python code on multiple CPUs in parallel with a single process (whereas multiprocessing runs multiple processes).
FTR, PEP 554 is explicitly independent of efforts to stop sharing the GIL between interpreters. I argue there that it is a good idea regardless. The existing functionality the PEP exposes, though, clearly benefits from better isolation between interpreters (including not sharing the GIL). :) On Thu, Nov 14, 2019 at 4:12 AM Victor Stinner <vstinner@python.org> wrote:
Another approach would be to pass a "PyContext*" pointer which contains tstate, but also additional fields. But I chose to state with a direct "PyThreadState* tstate" to avoid one indirection to every tstate access. Currently, tstate seems to be enough for the current code base.
FWIW, I favor this approach as well. As long as it is an opaque type, a PyContext allows us to be more flexible in adapting to the future. For now it could even be a simple alias for PyThreadState. Regardless, I'm not convinced that using a PyContext will have a real impact on runtime performance. Also, we already use "context" in a number of ways in Python. So "PyContext" might not be the best name. It probably needs to be a name without "context" in it or one with a concrete clue (e.g. 'PyRuntimeContext"). Anyway, thanks for driving this discussion, Victor! -eric