email@example.com (Martin v. Löwis) writes:
Wrong. If the code in B that calls Q does not allow threads, the callbacks don't need to reacquire the GIL.
I think you must be misunderstanding me. These callbacks might be invoked on threads that are not the Python main thread.
Which thread is the main thread is not relevant at all.
Sorry, I got "main" confused with "current". My point is that the callback may be invoked by threads that don't hold the GIL.
What matters is whether the callback is invoked in the context of the thread that installed it (i.e. as a result of calling a function in B).
I still don't see how the thread that installed it has any bearing. Imagine there's a global function pointer variable somewhere. Some thread comes along and makes it point to some function f ("installs the callback"). Now there are various ways that callback can be called. Some other thread may pick up the global variable at any time and invoke the function it points to. Why does it matter which thread set the variable?
Can you please explain what a callback is?
I'm trying to leave out irrelevant details, but a callback happens to be a virtual function in a C++ class instance in this case. These callbacks implement behaviors of base classes supplied by the library, Qt.
Ok, now I know how a callback is installed (by redefining a virtual method).
Technically, redefining a virtual function by itself doesn't do anything. You have to make an instance of the class which redefines that function available to the library somehow. But you knew that.
The other question is: How is it invoked? I.e. who invokes it, and why?
I suppose the immediate answer is "the library Q". However, that library does not invoke it with out a trigger: What is that trigger?
There are several ways, IIUC. It may be invoked in response to direct calls into Q's API (which may be made from a Python extension module). It may also be invoked by some thread that Q has launched.
What matters, AFAICT, is that the callback might be invoked on a thread that's not Python's main thread, thus must acquire the GIL, so if Python's main thread wants to call something in Q that might invoke one of these callbacks, it must release the GIL.
Which thread is the main thread is completely irrelevant. If you have something like
class MyB(B.Base): def overridden(self): print "This overrides a virtual function"
def doit(): b = MyB() B.do_the_real_work(b) # will call Q, which will call the callback, # which will call overridden
then there is no need for the C++ code in B to release the GIL, nor is there a need for B to reacquire the GIL in the callback. This is independent of whether doit is called in the main thread, or in the context of any other thread: at the point where do_the_real_work is called, the current thread holds the GIL.
Yes, we understand that problem. I had this exact discussion with the designer of B. He explained that the problem is that Q might also invoke the virtual function on a thread that is not holding the GIL.
If it keeps the GIL, then the GIL will be held at the point when the callback occurs, in the context of the thread where the callback occurs.
Maybe we have different ideas of what "callback" means. In your terms, it is not a "true callback".
Then I'm still curious as to what triggers invocation of that virtual function.
Q, either directly via its API, or in some thread it started. -- David Abrahams firstname.lastname@example.org * http://www.boost-consulting.com Boost support, enhancements, training, and commercial distribution