[capi-sig] Question about proper GIL management for a language interoperability tool

[Adam Olsen]

On Jan 11, 2008 11:14 AM, Tom Epperly <tepperly at llnl.gov> wrote:
> In ceval.h where Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS are
> defined, it says "WARNING: NEVER NEST CALLS TO Py_BEGIN_ALLOW_THREADS
> AND Py_END_ALLOW_THREADS!!!".  This makes me wonder if there is an issue
> with Python calling itself  through Babel. Let's assume that Python is
> the main driver, and it calls a Babel wrapped Python method to calculate
> factorial or something. This will result in nested calls to
> PyGILState_Ensure/PyGILState_Release and
> Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS.

The comment is misleading.  It's referring to a literal nesting, in
one function:

int
foo(void)
{
    Py_BEGIN_ALLOW_THREADS
    PyGILState_Ensure();
    Py_BEGIN_ALLOW_THREADS
    do_something();
    if (failed) {
        Py_BLOCK_THREADS
        return -1;
    }
    Py_END_ALLOW_THREADS
    PyGILState_Release();
    Py_END_ALLOW_THREADS
    return 0;
}

Note the Py_BLOCK_THREADS in the middle.  It will only handle the
inner BEGIN/END pair, so the outer pair will never get undone.  Then
again, you'll never call PyGILState_Release() either, so this example
is inherently hosed.

Your approach, where you're calling different functions and they all
clean up after themselves, seems just fine.

>
> Python interactive shell (invokes foo.Factorial(3))
>  Py_BEGIN_ALLOW_THREADS (in pStub_foo_Factorial (case 2))
>    PyGILState_Ensure (in foo_Factorial (case 1) arg=3)
>      Py_BEGIN_ALLOW_THREADS (in pStub_foo_Factorial (case 2))
>        PyGILState_Ensure (in foo_Factorial (case 1) arg=2)
>          Py_BEGIN_ALLOW_THREADS (in pStub_foo_Factorial (case 2))
>            PyGILState_Ensure (in foo_Factorial (case 1) arg=1)
>            PyGILState_Release (in foo_Factorial (case 1) arg=1)
>          Py_END_ALLOW_THREADS (in pStub_foo_Factorial (case 2))
>        PyGILState_Release (in foo_Factorial (case 1) arg=2)
>      Py_END_ALLOW_THREADS (in pStub_foo_Factorial (case 2))
>    PyGILState_Release (in foo_Factorial (case 1) arg=3)
>  Py_END_ALLOW_THREADS (in pStub_foo_Factorial (case 2))
> Python interactive shell



-- 
Adam Olsen, aka Rhamphoryncus