Hi,
Sorry, I was super busy this month and so I wasn't able to reply to this thread earlier :-( I wrote notes of the quick chat we had with Eric Snow, Nick Coghlan and Pablo Galindo Salgado at during Pycon sprints:
https://pythoncapi.readthedocs.io/runtime.html
- We want to be able to run multiple interpreters at the same time: reduce shared state, have one "interpreter lock" per interpreter (remove the *Global* Interpreter Lock, GIL)
- Currently, _PyRuntime variable is the root to get the current Python thread state. IMHO that has to change... but I'm not sure how :-) At least, each interpreter must have its private "gilstate".
- IMHO the Python thread state "tstate" must be the "root" to access basically everything: get the interpreter state (tstate->interp), get a module state, get the current exception, etc.
- The **public** C API must not be modified
- We can add new internal C API where we can pass as much "context" as we want
I started to modify Python **internals** to pass "runtime" and "tstate" in core files: pystate.c, ceval.c, pyerrors.c, signalmodule.c, pylifecycle.c, coreconfig.c, gcmodule.c, ...
=> https://bugs.python.org/issue36710
IMHO the "runtime" must be replaced with "tstate" in the long term. That's why I started to not pass "runtime" but directly pointers to fields inside runtime, so these functions will not have to be modified. Example:
void _PyEval_SignalReceived(struct _ceval_runtime_state *ceval);
Function called in signalmodule.c using:
_PyRuntimeState *runtime = &_PyRuntime; _PyEval_SignalReceived(&runtime->ceval);
If tomorrow, ceval is moved to PyInterpreterState, only signalmodule.c will have to be modified: not signalmodule.c
... Maybe passing directly "ceval" is too specific, and passing "tstate" everything will be fine in the future.
_PyRuntime should only be used to store the list of interpreters and anything to communicate between interpreters.
Victor