[Python-Dev] GIL removal question

[Antoine Pitrou]

On Sat, 13 Aug 2011 09:08:16 -0400
Guido van Rossum <guido at python.org> wrote:
> 
> And, though mostly off-topic, the worst problem with C code, calling
> back into Python, and the GIL that I have seen (several times):
> Suppose you are calling some complex C library that creates threads
> itself, where those threads may also call back into Python. Here you
> have to put a block around each Python callback that acquires the GIL
> before and releases it after, since the new threads (created by C
> code) start without the GIL acquired. I remember a truly nasty
> incident where the latter was done, but the main thread did not
> release the GIL since it was returning directly to Python (which would
> of course release the GIL every so many opcodes so the callbacks would
> run). But under certain conditions the block with the
> acquire-release-GIL code around a Python callback was invoked in the
> main thread (when a validation problem was detected early), and since
> the main thread didn't release the GIL around the call into the C
> code, it hung in a nasty spot. Add many layers of software, and a
> hard-to-reproduce error condition that triggers this, and you have a
> problem that's very hard to debug...

These days we have PyGILState_Ensure():
http://docs.python.org/dev/c-api/init.html#PyGILState_Ensure

and even dedicated documentation:
http://docs.python.org/dev/c-api/init.html#non-python-created-threads

;)

Regards

Antoine.