Embedding Python in a multi-threaded C++ application

[Mustafa Demirhan]

Jeff thanks for the prompt reply.

I know it is not a good idea to terminate a thread and it is best to
avoid it. But sometimes you may need it. And also it seems like
something similar to PyErr_SetInterrupt would be the best option.

Lets say I won't terminate a thread that is running, but I simply want
to stop it: As far as I see, even this is not possible. In order to
tell Python to stop running something, I need to do it from the worker
thread. However this does not seem possible to me. For example, in my
model, a worker thread creates a new thread state and then loads a
python module and runs a single method from that module. The thread
execution will not go to the next line until the python finishes
running that method. So there is no way for the worker thread to call
PyErr_SetInterrupt before the method already finishes.

Also, if done properly, I dont think that stopping a running Python
code will break anything. For example, lets say I have a Python code
that looks like

def myfun ():
    for x in xrange (100000):
        dosomething ()

Now, if there is  a way to fire an exception from the C code (like
PyErr_SetInterrupt does), the code will simply exit myfun without
breaking anything. The exception will be passed to higher levels and
then will be caught by the C code itself i think.

So, even a simple signaling mechanism would solve the problem in this
case. However, the only problem here is that the signal must be sent
from the main thread - not the working thread itself. My question is,
is there such a mechanism? :))

Looking forward to hearing from you soon.

Best wishes
Mustafa Demirhan

On Tue, 9 Nov 2004 15:50:13 -0600, Jeff Epler <jepler at unpythonic.net> wrote:
> Something along the lines of PyErr_SetInterrupt() is the best you can
> do in CPython, unless there's a facility I don't know about.  If your
> code is not running through the Python bytecode interpreter loop, then
> it's in some unknown and uninterruptable state.  I'm there may be
> something else you can do, but it'll be a "you get to keep both pieces
> [when it breaks]" kind of solution.
> 
> For some reasons why a facility to forcibly terminate a thread was
> removed from Java, a language with a richer set of thread-related tools,
> see
>     http://g.oswego.edu/dl/cpj/cancel.html
> down at the heading "Asynchronous Termination":
>     The stop method was originally included in class Thread, but its
>     use has since been deprecated. Thread.stop causes a thread to
>     abruptly throw a ThreadDeath exception regardless of what it is
>     doing. (Like interrupt, stop does not abort waits for locks or IO.
>     But, unlike interrupt, it is not strictly guaranteed to abort wait,
>     sleep, or join.)
> 
>     This can be an arbitrarily dangerous operation. Because Thread.stop
>     generates asynchronous signals, activities can be terminated while
>     they are in the midst of operations or code segments that
>     absolutely must roll back or roll forward for the sake of program
>     safety and object consistency.
> The same considerations apply to Python, and additionally Python has
> been designed on the side of simplicity and portability rather than to
> offer all the features of some particular OS threading library.
> (that said, PyErr_SetInterrupt *is* a lot like ThreadDeath, because it
> can be fired at any moment the interpreter "between bytecodes", which
> means all kinds of higher-level consistency requirements could be
> violated)
> 
> Jeff
> 
> 
> 


-- 
Mustafa Demirhan