On Thu, 13 Sep 2018 at 16:40, Victor Stinner firstname.lastname@example.org wrote:
While I clearly see an advantage of passing explicitly ctx to all functions of the C API, I also see that as a burden. I understood that it's not only calls to the C API that should be modified, but basically all C functions of CPython and all C functions of all C extensions, since ctx always comes from the parent. You would have to propagate ctx to every single C function which indirectly calls a function of the C API.
Plus you'd somehow have to pass it through any invoked Python code that subsequently calls back in to a C API.
It's not easy to justify this burden for an application which uses a single thread and a single interpreter.
Even without that concern, I don't see a way to handle the C -> Python -> C case without some form of implicit thread-local state management.
So yeah, if it would be possible to make ctx implicit... that would be better :-)
PEP 406 was a more constrained proposal to move just the import system state to an explicit "Import Engine" object, and even in that more limited case we quickly abandoned the idea of requiring that the engine always be passed to import hooks explicitly - it simply required too many changes to already standardised PEP 302 interfaces.
Fortunately, PEP 567 gives us some solid C API friendly context management primitives in the general case, and Eric's work on the subinterpreters API should hopefully finally get us to a point where the EnsureGIL APIs and subinterpreters actually play nice together.