Multithreaded C API Python questions
svein at seldal dot com
Fri Nov 17 00:22:20 CET 2006
> Forget all the low level PyGIL... functions.'
Quite the contrary, I would say!
From the Py C API doc, I interpret the PyGIL... functions to be higher
leveled than eg. PyEval_SaveThread(). I've checked into the source of
python to find out what's really going on, and I've learnt a couple of
interesting facts. The PyGIL... are helper functions build over the
PyEval... functions, not the other way around.
A real beauty about the PyGILState_Ensure() is the fact that if it's
called from a thread that is (yet) unknown to python, it will actually
create the thread state object for you (with PyThreadState_New) and
acquire the lock. Hence, I dont have to worry about the process of
swapping thread states with PyEval_xxxThread() functions. I ONLY need to
use PyGILState_Ensure() prior to any py-ops.
From my previous post, I asked about the difference between
PyEval_ReleaseLock() and PyEval_ReleaseThread() and I've found one major
difference. Both release the GIL as the docs states, but the
PyEval_ReleaseThread() saves the current thread state as well. When I
used PyEval_ReleaseLock() this didnt happen, preventing proper saving of
the current thread, causing py crash when control were handed back to
PyThreadState *pts = PyGILState_GetThisThreadState();
And for each py-op later on (from arbitrary thread):
gstate = PyGILState_Ensure();
Then you're home free. Everything related to threading is handled by
> Probably you just do a PyRun_xxxx in main thread and then everything
> else in Python, and expose C-parts for the thread-loop to Python as
> function (in other_py_inits) - where in the c-function you probably have
> the usual Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS bracket during
> time consuming/system/reenter-endangered message stuff.
The app i'm building is a plugin to a server system, and I cant control
nor remove any threads that the server uses. I am given one main thread
to run python forever, and messages that are to be delivered into python
are called from another.
I could do it like you propose: The data coming from my server will
arrive into a c-function called by a server thread. The c-part of the
thread-loop would then be run as another thread (started from python).
Its fully feasible, yet the challenge is to make proper data sync-ing
between these two threads.
Well, I think I've achieved what I wanted. Python seems apparently
stable, and reading from the python sources, I cant see any immediate
reasons why it shouldn't.
Thanks for letting me closer to a working solution!
More information about the Python-list