[Python-Dev] Extension modules, Threading, and the GIL

[David Abrahams]

martin@v.loewis.de (Martin v. L=F6wis) writes:

> David Abrahams <dave@boost-consulting.com> writes:
>
>> > So Q creates new threads which perform callbacks? But Q also performs
>> > the callbacks when invoked from A? Sounds like a bug in Q to me...
>>=20
>> Why do you say that?  Q doesn't know anything about Python or its
>> constraints.  Why should it be prohibited from invoking these
>> callbacks in whatever way it deems appropriate for its problem
>> domain?
>
> It is moderately evil for a library to create threads "under the
> hoods", IMO; in some domains, that might be a reasonable thing to
> do, provided there is a way for the application author to manage the
> threads on a higher level (e.g. by limiting the total number of
> threads that the library can create simultaneously).

I am not intimately familiar with Qt; these threads may not in fact be
created "under the hood".  Whether they are or not is IMO irrelevant
to the problem we're having, because it's not how the thread is
started that matters.

> If a library is creating new threads and invokes application code in
> these threads, threading should follow a threading model. That
> threading model has to be described, so that every application
> author can rely on certain features. The threading model is part of
> the library interface, just like the API.
>
> It appears that Q has no threading model. That is truly evil.

Though I am not intimately familiar with Qt, I can assure you that it
*does* have a threading model.

> In some cases, combining libraries with different threading models
> just won't work. For example, I recently found that Tcl's appartment
> threading model isn't really compatible with Python's GIL. It is
> possible to achieve interworking to some respect, but there are
> limitations that just can't be overcome (e.g. you just cannot invoke
> event dispatching in a thread that hasn't originally create the Tcl
> interpreter).
>
> If the threading model of Q is unknown or undefined, you cannot expect
> any kind of interworking.
>
>> I think rather that this is a library design which doesn't
>> interoperate well with Python's constraints on GIL manipulation.
>
> It seems to me that there is no design in the library

I think that judgement is at best premature.

> and this is the cause for the interoperability problem (or,
> perhaps, you just haven't presented the design).

No, I haven't.  I'm not very familiar with it myself.  I'm just
relating information I've got from the author of PyQt, who is very
familiar with it.

>> In this case, as Python is intended to be good for general
>> interoperability, it seems like Python ought to budge if possible.
>
> As Tim explains, this might not be possible.

It sounded to me like Tim explained that it is possible but
unimplemented.

--=20
                       David Abrahams
   dave@boost-consulting.com * http://www.boost-consulting.com
Boost support, enhancements, training, and commercial distribution