[Python-Dev] GIL vs thread state
Guido van Rossum
Mon, 14 Apr 2003 11:18:07 -0400
> > If pts->frame is not NULL, I'd expect a warning from
> > PyThreadState_Clear(): "PyThreadState_Clear: warning: thread still has
> > a frame\n".
> You mean this code, from Python/pystate.h?
> PyThreadState_Clear(PyThreadState *tstate)
> if (Py_VerboseFlag && tstate->frame != NULL)
> "PyThreadState_Clear: warning: thread still has a frame\n");
> Py_VerboseFlag is 0 set in my case, so no warning is printed.
> > > "Fatal Python error: PyThreadState_Get: no current thread" in the call
> > > to PyThreadState_Clear().
> > That's strange, because I cannot trace the code in there to such a
> > call. (Unless it is in a destructor.
> It is in a destructor: frame_dealloc, called from ZAP(tstate->frame).
Aha. That wasn't obvious from your description.
> > Can you tell more about where the PyThreadState_Get() call was?)
> This function allocates the threadstate for me:
> static void EnterPython(char *msg)
> PyThreadState *pts;
> pts = PyThreadState_New(g_interp);
> if (!pts)
> Py_FatalError("wincall: Could not allocate ThreadState");
> if (NULL != PyThreadState_Swap(pts))
> Py_FatalError("wincall (EnterPython): thread state not == NULL?");
Maybe you should have a look at Mark Hammond's PEP 311. It describes
the problem and proposes a better solution. (I think it requires you
to always use the existing thread state for the thread, rather than
making up a temporary thread state as is currently the idiom.)
> To explain the picture a little better, here is the sequence of calls:
> Python calls into a C extension.
> The C extension does
> The call_a_C_function calls back into C code like this:
> void MyCallback(void)
> EnterPython(); /* acquire the lock, and create a thread state */
> LeavePython(); /* destroy the thread state, and release the lock */
> Now, the execute_some_python_code() section is enclosed in a win32
> structured exception handling block, and it may return still with a
> frame in the threadstate, as it seems.
Ouch! I don't know what structured exception handling is, but this
looks like it would be as bad as using setjmp/longjmp to get back to
right after execute_some_python_code(). That code could leak
arbitrary Python references!!!
> Oops, I just tried the code in CVS python, and the problem goes away.
> Same for 2.3a2.
I vaguely recall that someone fixed some things in this area... :-(
> But my code has to run in 2.2.2 as well...
If the docs are lying, they have to be fixed. This is no longer my
prime area of expertise... :-(
--Guido van Rossum (home page: http://www.python.org/~guido/)