"Martin v. Löwis" <martin@v.loewis.de> writes:
David Abrahams wrote:
Python extension module B calls shared library Q and uses Q's callback interface. Because some of the callbacks need to use the Python API, and *might* be invoked by threads, they must all acquire the GIL.
Wrong. If the code in B that calls Q does not allow threads, the callbacks don't need to reacquire the GIL.
I think you must be misunderstanding me. These callbacks might be invoked on threads that are not the Python main thread. http://www.python.org/doc/current/api/threads.html says: "the rule exists that only the thread that has acquired the global interpreter lock may operate on Python objects or call Python/C API functions" I am taking that statement at face value and assuming it means what it says. Note that these threads were not started by Python's thread API.
Problem: using B while A is loaded breaks A: because B has installed callbacks in Q that acquire the GIL, A must also release the GIL before calling into Q.
Can you please explain what a callback is?
I'm trying to leave out irrelevant details, but a callback happens to be a virtual function in a C++ class instance in this case. These callbacks implement behaviors of base classes supplied by the library, Qt. For the purposes of this discussion, it might just as well be a 'C' language pointer-to-function, though: void (*)()
Can the callbacks occur in the context of a different thread, i.e. different from the one that has installed the callback?
I think the answer is yes, but I'm trying to leave out the irrelevant, and I'm pretty sure that it doesn't matter one whit which thread installs the callback. What matters, AFAICT, is that the callback might be invoked on a thread that's not Python's main thread, thus must acquire the GIL, so if Python's main thread wants to call something in Q that might invoke one of these callbacks, it must release the GIL.
If it is a true callback (i.e. Q will call B back while being called from B), then this won't interfere at all with A.
Maybe we have different ideas of what "callback" means. In your terms, it is not a "true callback". -- David Abrahams dave@boost-consulting.com * http://www.boost-consulting.com Boost support, enhancements, training, and commercial distribution