Jeremy Maxfield <anothermax@gmail.com> writes:
The current threadmodule.c does not seem to correctly support multiple (sub) interpreters.
This would seem to be an accurate statement. A short history: The GILState functions were implemented. The way they work is that when you call PyGILState_Ensure, (essentially) a thread local variable is checked to see if a thread state is known for this thread. If one is found, fine, it is used. If not, one is created (using the interpreter state what was passed to _PyGILState_Init() by Py_Initialize()) and stored in the thread local variable. This had a (pretty obvious in hindsight) problem (bug #1010677): when you create a thread via thread.start_new_thread a thread state is created, but not stored in the thread local variable consulted by PyGILState_Ensure. So if you call PyGILState_Ensure another thread state is created for the thread (generally a no-no) and if you already hold the GIL PyGILState_Ensure will attempt to acquire it again -- a deadlock. This was fixed by essentially using PyGILState_Ensure to create the threadstate. This has a (pretty obvious in hindsight) problem (bug #1163563): PyGILState_Ensure always uses the interpreter state created by Py_Initialize, ignoring the interpreter state carefully supplied to t_bootstrap. Hilarity ensues. So, what's the fix? Well, the original problem was only the lack of association between a C level thread and a thread state. This can be fixed by setting up this association in t_bootstrap (and I've posted a patch that does this to the report of #1163563). This suffices for all known problems, but it's a bit hackish. Another approach is to set up this association PyThreadState_New(), which is possibly a bit more elegant, but carries a risk: is PyThreadState_New always called from the new thread? ISTM that it is, but I'm not sure. I'm not expecting anyone else to think hard about this on recent form, so I'll think about it for a bit and then fix it in the way that seems best after that. Feel free to surprise me. Cheers, mwh -- I would hereby duly point you at the website for the current pedal powered submarine world underwater speed record, except I've lost the URL. -- Callas, cam.misc