[Python-Dev] Extension modules, Threading, and the GIL

[David Abrahams]

"Martin v. L=F6wis" <martin@v.loewis.de> writes:

> David Abrahams wrote:
>> IIUC, that strategy doesn't get Mark what he wants in this case:
>>     2.2) Extensions that *may* take a little time, but more to the
>>     point, may directly and synchronously trigger callbacks.  That is,
>>     it is not expected that much time will be spent outside of Python,
>>     but rather that Python will be re-entered.
>> Which is to be able to avoid releasing the GIL in the case where the
>> extension isn't going to do much other than invoke the callback
>> function which re-acquires it.
>
> I think you are incorrectly interpreting Mark's priorities:
>
>     I am not looking for a change in semantics, just a
>     simple way to do it (and maybe even a fast way to do it,
>     but that is secondary).
>
> So performance is not the his primary goal. The goal is that it is
> easy to use, and I think my strategy is fairly easy to follow: If in
> doubt, release the lock.

OK.  I guess there's one more point worth mentioning: APIs are not
always scrupulously documented.  In particular, documentation might
give you no reason to think any callbacks will be invoked for a given
call, when in fact it will be.  Furthermore, problems with not
releasing the GIL will don't show up with arbitrary callbacks in the
API, only when someone finally installs one which uses Python's API.

The Windows API is a prime example of this, but I'm sure there are
many others.  If we could make "creating a thread state and acquiring
the current thread" immune to the condition where the the current
thread is already acquired, we'd be making it much easier to write
bulletproof extensions.

--=20
                       David Abrahams
   dave@boost-consulting.com * http://www.boost-consulting.com
Boost support, enhancements, training, and commercial distribution