[Python-Dev] Extension modules, Threading, and the GIL

[Guido van Rossum]

> We do have a real problem here, and I keep stumbling across it.  So far,
> this issue has hit me in the win32 extensions, in Mozilla's PyXPCOM, and
> even in Gordon's "installer".  IMO, the reality is that the Python external
> thread-state API sucks.  I can boldly make that assertion as I have heard
> many other luminaries say it before me.  As Tim suggests, time is the issue.
> 
> I fear the only way to approach this is with a PEP.  We need to clearly
> state our requirements, and clearly show scenarios where interpreter states,
> thread states, the GIL etc all need to cooperate.  Eg, InterpreterState's
> seem YAGNI, but manage to complicate using ThreadStates, which are certainly
> YNI.  The ability to "unconditionally grab the lock" may be useful, as may a
> construct meaning "I'm calling out to/in from an external API" discrete from
> the current singular "release/acquire the GIL" construct available today.
> 
> I'm willing to help out with this, but not take it on myself.  I have a fair
> bit to gain - if I can avoid toggling locks every time I call out to each
> and every function there would be some nice perf gains to be had, and
> horrible code to remove.

I welcome a PEP on this!  It's above my own level of expertise, mostly
because I'm never in a position to write code that runs into this...

--Guido van Rossum (home page: http://www.python.org/~guido/)