[Python-Dev] Extension modules, Threading, and the GIL

[David Abrahams]

martin@v.loewis.de (Martin v. L=F6wis) writes:

> David Abrahams <dave@boost-consulting.com> writes:
>
>> I am not intimately familiar with Qt
>
> Ah, so Q is Qt :-)=20

I actually revealed that several messages back.

> Care to reveal what A and B are

A is some extension module written by one of my users, that calls Qt
but doesn't install any callbacks.

B is some extension module which uses PyQt, thus installs callbacks
which invoke Python.

> and which the callback is that caused a deadlock?

There was no deadlock, as I've said.  The symptom is that Python
complains at some point that there's no thread state.  It goes away if
A releases the GIL before calling into Qt, and reacquires the GIL
afterwards.  I speculate that the callback releases the GIL when it is
finished, so that when A returns to Python there is no current thread.
By that time, the callback has completed, so it's hard to know for
sure which one it was.

>> It sounded to me like Tim explained that it is possible but
>> unimplemented.
>
> Trying to channel Tim: All experts for this stuff have tried and
> failed; Mark Hammond has a sort-of solution which Tim believes to be
> strictly-speaking incorrect. While people may have thought they have
> a solution, it is troubling that nobody can remember what the
> solution is. Maybe the email signature just had not enough space to
> write it down :-)

Hmm, it seems as though a mutex-protected record of which thread is
currently holding the GIL should be enough to handle it.  There must
be subtle details which complicate the problem, e.g. that there's no
portable/reliable way to identify the current thread (?)

--=20
                       David Abrahams
   dave@boost-consulting.com * http://www.boost-consulting.com
Boost support, enhancements, training, and commercial distribution