"Mark Hammond" <mhammond@skippinet.com.au> writes:
We do have a real problem here, and I keep stumbling across it. So far, this issue has hit me in the win32 extensions, in Mozilla's PyXPCOM, and even in Gordon's "installer". IMO, the reality is that the Python external thread-state API sucks. I can boldly make that assertion as I have heard many other luminaries say it before me. As Tim suggests, time is the issue.
I fear the only way to approach this is with a PEP. We need to clearly state our requirements, and clearly show scenarios where interpreter states, thread states, the GIL etc all need to cooperate. Eg, InterpreterState's seem YAGNI, but manage to complicate using ThreadStates, which are certainly YNI. The ability to "unconditionally grab the lock" may be useful, as may a construct meaning "I'm calling out to/in from an external API" discrete from the current singular "release/acquire the GIL" construct available today.
I'm willing to help out with this, but not take it on myself. I have a fair bit to gain - if I can avoid toggling locks every time I call out to each and every function there would be some nice perf gains to be had, and horrible code to remove.
I'm also willing to lend a hand with a PEP, if it's worth anything. I don't know as much about the problems in this domain as you do; I've only seen this one example that bit me. I'm prepared to spend a few brain cycles on it and help with the writing, though. -- David Abrahams dave@boost-consulting.com * http://www.boost-consulting.com Boost support, enhancements, training, and commercial distribution