Multithreaded C API Python questions

[robert]

Svein Seldal wrote:
> robert wrote:
>> PyGILState_Ensure/Release guarantees to establish the GIL - even if it 
>> was already held (useful if you deal with complex call 
>> ordering/dependencies)
> 
> I understand this to mean that I dont need to explicitly lock the GIL 
> (with PyEval_AcquireLock() or PyEval_AcquireThread()).
> 
> Well I cant figure out the PyGILState_Ensure() because if I only use 
> this function to establish the GIL, when calling python, python will 
> shortly after crash with "Fatal Python error: ceval: tstate mix-up". 
> This happens consistently when the main app and the extra thread has 
> called python and both are busy executing python code.

Do did't tell enough to see what you want to do on the big picture.

usually you create a thread through Python.
If you boot Python at all from C you have to do 
PyEval_InitThreads 
or maybe more simple use the high level layer PyRun_...

If you boot a new Python thread manually or a separated interpreter (with separated module state) from outside 
the you have to also do:

(PyInterpreterState* PyInterpreterState_New() ) 
PyThreadState* PyThreadState_New( PyInterpreterState *interp) 
void PyEval_AcquireThread( PyThreadState *tstate) 


If you are already in a Python thread but don't for some reason not know the current thread state:
PyThreadState* PyThreadState_Get( ) 
but usually you release with PyThreadState* PyEval_SaveThread( ) / Py_BEGIN_ALLOW_THREADS 



>> PyObject_CallObject(...)     calls Python code. Thus the interpreter 
>> will switch and do as usuall during that.
> 
> BTW What do you mean by switch?

the automatic scheduling every sys.getcheckinterval() etc.. you mentioned
You'd only have to take care for that, if your C-code does things (e.g. calling python basic funcs) for a long time and doesn't release the lock with Py_BEGIN_ALLOW_THREADS ...

> The main problem is that not done this way, it's the other way around. 
> My main C app will call a python function which will be a lengthy time 
> consuming process.
> 
> The main C app will call python and it will never return from 
> PyObject_CallObject(...), while the extra thread should call a py 
> function to deliver occational messages into C.

Maybe simply boot Python in the main thread (Py_Initialize( )) and run off Python possibly as simple as PyRun_String 
and let Python do thread.start_new(...)
You'd not have to worry much about states.

( and later you'd probably even more simple do "python myapp.py"  and expose your C app as extension module :-) )


-robert