On Tue, 21 Apr 2020 12:05:28 -0700 "Gregory P. Smith" <greg@krypto.org> wrote:
On Tue, Apr 21, 2020 at 10:49 AM Antoine Pitrou <solipsis@pitrou.net> wrote:
On Tue, 21 Apr 2020 18:46:04 +0200 Petr Viktorin <encukou@gmail.com> wrote:
On 2020-04-21 11:01, Antoine Pitrou wrote:
On Mon, 20 Apr 2020 19:21:21 -0600 Eric Snow <ericsnowcurrently@gmail.com> wrote:
Honest question: how many C extensions have process-global state that will cause problems under subinterpreters? In other words, how many already break in mod_wsgi?
A slightly tricky question is what happens if a PyObject survives longer than the subinterpreter that created it.
For example, in PyArrow, we allow passing a Python buffer-like object as a C++ buffer to C++ APIs. The C++ buffer could conceivably be kept around by C++ code for some time. When the C++ buffer is destroyed, Py_DECREF() is called on the Python object (I figure that we would have to switch to the future interpreter-aware PyGILState API -- when will it be available?).
In this scenario given that the PyObject owning the buffer passed to C++ by PyArrow was Py_INCREF'd, an option during Py_Finalize is to not release anything who's refcount never made it to 0. Which also implies leaving enough of the interpreter state around so that Py_DECREF could be called and trigger the free().
That's assuming a trivial destructor. Of course, that would cater for most concrete use cases. Regards Antoine.