Questions on the C Thread API

[Phil Thompson]

I'm trying to properly implement the Qt threading classes in PyQt so
that they correctly interact with Python's thread support, and I'm
having a problem getting my head around some of the issues.

In a PyQt script, execution flips between Python and C++. A script will
call a C++ method, which may call a C++ virtual method that has been
re-implemented in Python, which itself calls another C++ method, etc,
etc. The boundaries points between Python and C++ have to manage the
global lock and the current thread state to allow other (Python) threads
to run.

Currently, any C++ method that might block effectively has a call to
PyEval_SaveThread() before it and a call to PyEval_ReleaseThread() after
it. Any C++ virtual method that has been re-implemented in Python first
checks to see if it already has the global lock, and acquires it if it
doesn't. The Python method is then evaluated, and the global lock
released if it had previously been acquired. The check if the lock has
already been acquired is necessary because I only release the lock on a
small subset of C++ methods, ie. those that might block, which I do for
reasons of efficiency.

All the above works fine, so long as all the C++ calls are made from the
same thread. With the support of the Qt thread classes, that assumption
is no longer valid.

Therefore, just before calling the Python code that re-implements a C++
virtual method, the global lock has to be acquired, and the correct
thread state restored. However, to prevent a deadlock the thread state
can only be restored if it is not the current one, but there is no way
to get the value of the current thread state without having acquired the
global lock. Catch 22.

The obvious solution (I think) is to always release the global lock when
entering C++ code and always acquire it again when entering Python code
- but as only a couple of dozen out of over 5000 methods are affected
I'd like to avoid it if possible.

Is my thinking correct here? Are there other alternatives?

Thanks,
Phil