[Tim]
[David Abrahams]
... but AFAICT there _is_ a Python core issue here: there's no way to find out whether you've already got the GIL**, so if you _might_ have to acquire it, you must always acquire it.
**If I'm wrong about that, please let me know. It isn't obvious from the documentation.
It's true -- you can't know whether you have the GIL, unless you code up another layer of your own machinery to keep track of who has the GIL. Mark Hammond faces this issue (in all its multifacted glories) in the Win32 extensions, and built some C++ classes there to help him out. It's difficult at best, and last I looked (several years ago) I wasn't convinced.
We do have a real problem here, and I keep stumbling across it. So far, this issue has hit me in the win32 extensions, in Mozilla's PyXPCOM, and even in Gordon's "installer". IMO, the reality is that the Python external thread-state API sucks. I can boldly make that assertion as I have heard many other luminaries say it before me. As Tim suggests, time is the issue. I fear the only way to approach this is with a PEP. We need to clearly state our requirements, and clearly show scenarios where interpreter states, thread states, the GIL etc all need to cooperate. Eg, InterpreterState's seem YAGNI, but manage to complicate using ThreadStates, which are certainly YNI. The ability to "unconditionally grab the lock" may be useful, as may a construct meaning "I'm calling out to/in from an external API" discrete from the current singular "release/acquire the GIL" construct available today. I'm willing to help out with this, but not take it on myself. I have a fair bit to gain - if I can avoid toggling locks every time I call out to each and every function there would be some nice perf gains to be had, and horrible code to remove. Once I clear the mail from my break I will try and find the thread-sig conclusions... Mark.