Question about proper GIL management for a language interoperability tool
I working on a language interoperability tool called Babel, https://computation.llnl.gov/casc/components/. It provides high-performance, bi-directional communication between any combination of C, C++, F77, F90, Java, and Python. Right now, I am trying to make sure I am handling the GIL locking and unlocking correctly, and I am wondering if the Python community can confirm whether I am handling things appropriately.
Babel handles the many-to-many language interoperability problem with a hub and spoke architecture, so all inter-language calls go through C. Hence, I can discuss how I am handling the GIL in terms of communications between C and Python. I am going to describe what I am doing using pseudo-code and leaving out irrelevant details.
CASE 1 (some other language calling Python via C)
foo_Factorial(struct foo_t *self, int32_t arg) { /* C routine that calls Python */ PyGILState_STATE _gstate; _gstate = PyGILState_Ensure(); /* convert incoming arguments from C to Python */ _result = PyObject_CallObject(_pfunc, _args); /* process outgoing arguments or exceptions */ PyGILState_Release(_gstate); /* return to caller in whatever language */ }
CASE 2 (Python calling some other language via C)
static PyObject * pStub_foo_Factorial(PyObject *_self, PyObject *_args, PyObject *_kwdict) { PyObject *result; /* unpack incoming Python arguments into their C equivalents */ Py_BEGIN_ALLOW_THREADS /* dispatch to method implementation */ foo_Factorial(i,k,l); /* this step is actually done through a function pointer */ Py_END_ALLOW_THREADS /* pack outgoing C arguments into the Python return value or process any exceptions */ return result; }
Discussion
Case 1 seems pretty straight forward. The current thread must own the GIL before making any Python C API calls, and it should release it when it's done. Case 2 also seems appropriate because function x_y_Method could very well do blocking I/O operation. For example, Babel also supports remote method invocation (RMI), so x_y_Method might dispatch the method across the network to another machine.
In ceval.h where Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS are defined, it says "WARNING: NEVER NEST CALLS TO Py_BEGIN_ALLOW_THREADS AND Py_END_ALLOW_THREADS!!!". This makes me wonder if there is an issue with Python calling itself through Babel. Let's assume that Python is the main driver, and it calls a Babel wrapped Python method to calculate factorial or something. This will result in nested calls to PyGILState_Ensure/PyGILState_Release and Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS.
Python interactive shell (invokes foo.Factorial(3)) Py_BEGIN_ALLOW_THREADS (in pStub_foo_Factorial (case 2)) PyGILState_Ensure (in foo_Factorial (case 1) arg=3) Py_BEGIN_ALLOW_THREADS (in pStub_foo_Factorial (case 2)) PyGILState_Ensure (in foo_Factorial (case 1) arg=2) Py_BEGIN_ALLOW_THREADS (in pStub_foo_Factorial (case 2)) PyGILState_Ensure (in foo_Factorial (case 1) arg=1) PyGILState_Release (in foo_Factorial (case 1) arg=1) Py_END_ALLOW_THREADS (in pStub_foo_Factorial (case 2)) PyGILState_Release (in foo_Factorial (case 1) arg=2) Py_END_ALLOW_THREADS (in pStub_foo_Factorial (case 2)) PyGILState_Release (in foo_Factorial (case 1) arg=3) Py_END_ALLOW_THREADS (in pStub_foo_Factorial (case 2)) Python interactive shell
Is this approach correct?
Regards,
Tom Epperly LLNL/CASC
On 11-Jan-2008, at 19:14 , Tom Epperly wrote:
CASE 1 (some other language calling Python via C)
foo_Factorial(struct foo_t *self, int32_t arg) { /* C routine that calls Python */ PyGILState_STATE _gstate; _gstate = PyGILState_Ensure(); /* convert incoming arguments from C to Python */ _result = PyObject_CallObject(_pfunc, _args); /* process outgoing arguments or exceptions */ PyGILState_Release(_gstate); /* return to caller in whatever language */ }
This is what I always do too.
CASE 2 (Python calling some other language via C)
static PyObject * pStub_foo_Factorial(PyObject *_self, PyObject *_args, PyObject
*_kwdict) { PyObject *result; /* unpack incoming Python arguments into their C equivalents */ Py_BEGIN_ALLOW_THREADS /* dispatch to method implementation */ foo_Factorial(i,k,l); /* this step is actually done through a
function pointer */ Py_END_ALLOW_THREADS /* pack outgoing C arguments into the Python return value or process any exceptions */ return result; }
Here I use
PyThreadState *_save = PyEval_SaveThread();
...
PyEval_RestoreThread(_save);
But that pretty much amounts to the same, I think, modulo C
preprocessor trickery (which got in my way because I'm working in C++).
This module does serious multithreaded nesting from Python to C++ to
Python to C++ etc etc etc, and I haven't yet run into a problem.
Jack Jansen, Jack.Jansen@cwi.nl, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma
Goldman
On Jan 11, 2008 11:14 AM, Tom Epperly tepperly@llnl.gov wrote:
In ceval.h where Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS are defined, it says "WARNING: NEVER NEST CALLS TO Py_BEGIN_ALLOW_THREADS AND Py_END_ALLOW_THREADS!!!". This makes me wonder if there is an issue with Python calling itself through Babel. Let's assume that Python is the main driver, and it calls a Babel wrapped Python method to calculate factorial or something. This will result in nested calls to PyGILState_Ensure/PyGILState_Release and Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS.
The comment is misleading. It's referring to a literal nesting, in one function:
int foo(void) { Py_BEGIN_ALLOW_THREADS PyGILState_Ensure(); Py_BEGIN_ALLOW_THREADS do_something(); if (failed) { Py_BLOCK_THREADS return -1; } Py_END_ALLOW_THREADS PyGILState_Release(); Py_END_ALLOW_THREADS return 0; }
Note the Py_BLOCK_THREADS in the middle. It will only handle the inner BEGIN/END pair, so the outer pair will never get undone. Then again, you'll never call PyGILState_Release() either, so this example is inherently hosed.
Your approach, where you're calling different functions and they all clean up after themselves, seems just fine.
Python interactive shell (invokes foo.Factorial(3)) Py_BEGIN_ALLOW_THREADS (in pStub_foo_Factorial (case 2)) PyGILState_Ensure (in foo_Factorial (case 1) arg=3) Py_BEGIN_ALLOW_THREADS (in pStub_foo_Factorial (case 2)) PyGILState_Ensure (in foo_Factorial (case 1) arg=2) Py_BEGIN_ALLOW_THREADS (in pStub_foo_Factorial (case 2)) PyGILState_Ensure (in foo_Factorial (case 1) arg=1) PyGILState_Release (in foo_Factorial (case 1) arg=1) Py_END_ALLOW_THREADS (in pStub_foo_Factorial (case 2)) PyGILState_Release (in foo_Factorial (case 1) arg=2) Py_END_ALLOW_THREADS (in pStub_foo_Factorial (case 2)) PyGILState_Release (in foo_Factorial (case 1) arg=3) Py_END_ALLOW_THREADS (in pStub_foo_Factorial (case 2)) Python interactive shell
-- Adam Olsen, aka Rhamphoryncus
participants (3)
-
Adam Olsen -
Jack Jansen -
Tom Epperly