[Python-Dev] Extension modules, Threading, and the GIL

[Martin v. Löwis]

David Abrahams <dave@boost-consulting.com> writes:

> > What matters is whether the callback is invoked in the context of
> > the thread that installed it (i.e. as a result of calling a function
> > in B).
> 
> I still don't see how the thread that installed it has any bearing.

If the callbacks would only occur while Q is being called from B, and
in the context of the thread which also hosts that call, then B should
not release the lock. Then the callback would not need to reacquire
it.

> There are several ways, IIUC.  It may be invoked in response to direct
> calls into Q's API (which may be made from a Python extension module).
> It may also be invoked by some thread that Q has launched.

I find this surprising. Is this *any* API of Q that can trigger the
callbacks, or a dispatch_one_event call that can do so?

> Yes, we understand that problem.  I had this exact discussion with the
> designer of B.  He explained that the problem is that Q might also
> invoke the virtual function on a thread that is not holding the GIL.

I would more deeply question such a statement. It sounds like this
library uses the callback for event processing of some kind, in which
case the callbacks are only invoked if events are processed. It should
be possible to describe more precisely under what specific conditions
event processing occurs.

> > Then I'm still curious as to what triggers invocation of that virtual
> > function.
> 
> Q, either directly via its API, or in some thread it started.

Then calling this API must release the GIL.

Regards,
Martin