Hi, I'm developing a software which embeds Python as scripting language.
The software acts as webserver and for each request it peeks a thread
from a pool which in turn loads a python script to generate the response
(something like mod_python).
Since each http request is "independent" from others one, I need to
create a new interpreter with its own dictionary every time.
I'm exposing lots of C++ wrapped code and, in order to provide a sort of
"parallelism" between each request, I'm using this mechanism for each
registered callback:
P = python core
A = my application
P: invokes the registered C callback
A: PyEval_SaveThread (allows others threads to run/resume their scripts)
A: invoke C/C++ application's functions (which don't touch CPython API)
A: PyEval_RestoreThread (takes back the lock)
P: resume the execution of the script
(the above schema is the same of
Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS documented in ceval.h at
line 83)
The problem I have encountered is that when I process two requests
simultaneously, Python reports the fatal error "ceval: orphan tstate".
I'm not an expert of the CPython internals API but I take a look at the
file ceval.c (actually I'm using python 2x but in the 3x version I
noticed it's the same) and the code involved is:
/* Give another thread a chance */
if (PyThreadState_Swap(NULL) != tstate)
Py_FatalError("ceval: tstate mix-up");
PyThread_release_lock(interpreter_lock);
/* Other threads may run now */
PyThread_acquire_lock(interpreter_lock, 1);
if (PyThreadState_Swap(tstate) != NULL)
Py_FatalError("ceval: orphan tstate");
Can anyone explain me the meaning of those fatal errors (in particular
the "orphan tstate" one)? Why the return value should be null?
As far as I understand after the "PyThread_release_lock" others threads
are allowed to run and, if you take a look again at my above schema,
PyThreadState_Swap is supposed to be called between
PyThread_release_lock/PyThread_acquire_lock, exactly where the comment
"Other threads may run now" is placed. If i changed the code in order to
not fire a fatal error after the second PyThreadState_Swap everything
seems to work fine, but I'm afraid it's not the "proper" solution and I
need to understand the meaning of that fatal error.
Sorry for the long post and for my bad english, I hope that someone
could really help me.
Thanks
