Extension modules, Threading, and the GIL

hg hansgeunsmeyer at earthlink.net
Wed Jan 1 22:35:57 EST 2003


David Abrahams <dave at boost-consulting.com> wrote in message 

> Q == Qt, and apparently it may invoke some of the
> callbacks in response to calls to its API, and also in response to
> asynchronous events (e.g. on some other thread than the one Python is
> executing in).
> 
> > But while I'm not opposed to some brilliant fix to the thread
> > bootstrapping problems in Python, my simplistic answer would be that
> > external function calls should always release the interpreter lock,
> > whether they need to in anticipation of callbacks or not.  
> 
> Interesting; that's what the author of B said as well.  However, most
> of my users have absolutely no need to release the GIL, and I'd hate
> to make them pay for it just because a few strange applications do.

> > If implementation A fails to do this and it causes problems for B,
> > then it's time to fix A.
> 
> Clearly, that's the only available solution at present.

Releasing the GIL in A before calling into Q would be one solution,
perhaps the most principled one.  But you might just as well say that
B needs to be fixed.

If it's possible to install a callback function in Q that can be
called either
synchronously or asynchronously, and if the implementation of this
callback
in B will _always_ try to acquire the GIL before it calls into Python
core code,
then I'd say that B needs to be fixed.  B should not unconditionally
try to acquire the GIL. Even though the code in B may have no way to
determine if it (the current thread) has the GIL, it can see if there
is a current Python thread state at all (PyThreadState_GET).  If there
is one, the function is called synchronously; if there isn't, it's
called asynchronously, and only in this case should it create (or
retrieve) its own PyThreadState, and acquire the GIL. (It's possible
that such a callback could be called after the Python




More information about the Python-list mailing list