On Tue, Apr 21, 2020 at 3:10 AM Antoine Pitrou <solipsis@pitrou.net> wrote:
On Mon, 20 Apr 2020 19:21:21 -0600 Eric Snow <ericsnowcurrently@gmail.com> wrote:
Honest question: how many C extensions have process-global state that will cause problems under subinterpreters? In other words, how many already break in mod_wsgi?
A slightly tricky question is what happens if a PyObject survives longer than the subinterpreter that created it.
For example, in PyArrow, we allow passing a Python buffer-like object as a C++ buffer to C++ APIs. The C++ buffer could conceivably be kept around by C++ code for some time. When the C++ buffer is destroyed, Py_DECREF() is called on the Python object (I figure that we would have to switch to the future interpreter-aware PyGILState API -- when will it be available?). But what happens if the interpreter which created the Python object is already destroyed at this point?
Good question (and example). I'm not familiar enough with the GC machinery to have a good answer. I do know that GC is per-interpreter now, but the allocators are still global. I'm not sure how object lifetime is *currently* tied to interpreter lifetime. I'd expect that, at the point we have per-interpreter allocators, that all objects in that interpreter would be destroyed when the interpreter is destroyed. -eric