[Python-Dev] Signals, threads, blocking C functions
gjcarneiro at gmail.com
Mon Sep 4 16:52:36 CEST 2006
On 9/4/06, Nick Maclaren <nmm1 at cus.cam.ac.uk> wrote:
> "Gustavo Carneiro" <gjcarneiro at gmail.com> wrote:
> > I am now thinking of something along these lines:
> > typedef void (*PyPendingCallNotify)(void *user_data);
> > PyAPI_FUNC(void) Py_AddPendingCallNotify(PyPendingCallNotify callback,
> > void *user_data);
> > PyAPI_FUNC(void) Py_RemovePendingCallNotify(PyPendingCallNotify
> > callback, void *user_data);
> Why would that help? The problems are semantic, not syntactic.
> Anthony Baxter isn't exaggerating the problem, despite what you may
> think from his posting.
You guys are tough customers to please. I am just trying to solve a
problem here, not create a new one; you have to believe me.
OK, let's review what we know about current python, signals, and threads:
1. Python launches threads without touching sigprocmask;
2. Python installs signal handlers for all signals;
3. Signals can be delivered to any thread, let's assume (because
of point #1 and not others not mentioned) that we have no control over
which threads receive which signals, might as well be random for all
4. Python signal handlers do almost nothing: just sets a flag,
and calls Py_AddPendingCall, to postpone the job of handling a signal
until a "safer" time.
5. The function Py_MakePendingCalls() should eventually get
called at a "safer" time by user or python code.
6. It follows that until Py_MakePendingCalls() is called, the
signal will not be handled at all!
Now, back to explaining the problem.
1. In PyGTK we have a gobject.MainLoop.run() method, which blocks
essentially forever in a poll() system call, and only wakes if/when it
has to process timeout or IO event;
2. When we only have one thread, we can guarantee that e.g.
SIGINT will always be caught by the thread running the
g_main_loop_run(), so we know poll() will be interrupted and a EINTR
will be generated, giving us control temporarily back to check for
3. When we have multiple thread, we cannot make this assumption,
so instead we install a timeout to periodically check for signals.
We want to get rid of timeouts. Now my idea: add a Python API to say:
"dear Python, please call me when you start having pending calls,
even if from a signal handler context, ok?"
>From that point on, signals will get handled by Python, python calls
PyGTK, PyGTK calls a special API to safely wake up the main loop even
from a thread or signal handler, then main loop checks for signal by
calling PyErr_CheckSignals(), it is handled by Python, and the process
lives happily ever after, or die trying.
I sincerely hope my explanation was satisfactory this time.
PS: there's a "funny" comment in Py_AddPendingCall that suggests it is
not very safe against reentrancy problems:
/* XXX Begin critical section */
/* XXX If you want this to be safe against nested
XXX asynchronous calls, you'll have to work harder! */
Are signal handlers guaranteed to not be interrupted by another
signal, at least? What about threads?
More information about the Python-Dev