Le mar. 17 mars 2020 à 15:47, Mark Shannon <mark@hotpy.org> a écrit :
There is no PEP but scatted documents. I wrote a short article to elaborate the context of this work: https://vstinner.github.io/cpython-pass-tstate.html
One motivation is to ease the implementation of subinterpreters (PEP 554). But PEP 554 describes more than public API than the implementation.
I don't see how this eases the implementation of subinterpreters.
The overall work helps to identify what is currently shared by subinterpreters and prepares the code base to "unshare" data. For me, it's more explicit that _PyErr_Occurred(tstate) gets the current exception from the tstate argument, than PyErr_Occurred() which implicitly gets the current Python thread state. Before the work on subinterpreters started, it was really hard to guess where all the states come from and if anything was shared. I mean, tons of states were shared. Just because it's easier to implement like that! Eric Snow started by moving many core states into _PyRuntimeState or PyInterpreterState between Python 3.7 and Python 3.8. Previously, many functions had an implicit state which was shared by all interpreters. Now, it's still shared, but it becomes easier to reorganize the code to have one state per interpreter. For example, that's how I managed to move the state of the gc module per-interpreter: https://bugs.python.org/issue36854 To be able to run two interpreters in parallel, it's very important to not share anything for best performances but also for correctness. Concurrent accesses of shared states with no locking mecasim can introduce inconsistencies. One solution for that is to not share anything ;-)
I also opened a discussion on other singletons (None, True, False, ...): https://bugs.python.org/issue39511
That's great, but I don't see the relevance.
I proposed a fix which moves None and True singletons into PyInterpreterState. If we choose this solution, it would mean that more and more functions will have to pass tstate (ex: none = tstate->interp->none). But let's discuss the details in https://bugs.python.org/issue39511 Other solutions have been proposed ;-)
The long-term goal is to be able to run multiple isolated interpreters in parallel.
An admirable goal, IMO. But how is it to be done? That is the question.
Passing states explicitly in one task. Another task is to convert modules to the multiphase initialization PEP 489. It allows to have multiple isolated instances of the same C extension module, one per subinterpreter. No more shared module state! Eric Snow tracks the different tasks under the https://github.com/ericsnowcurrently/multi-core-python/ project.
Changes should only be made if necessary for correctness or for a significant improvement of performance.
It's for correctness ;-) Victor -- Night gathers, and now my watch begins. It shall not end until my death.