[New-bugs-announce] [issue24544] Race condition and crash when embedding multi-thread script

Oleksiy Markovets report at bugs.python.org
Wed Jul 1 15:05:29 CEST 2015

New submission from Oleksiy Markovets:


While embedding python script in c++ application I faced random crashes 
with error:

Fatal Python error: Py_EndInterpreter: not the last thread                                                                                                                                                                                                                     

Even though I explicitly join each thread created with *threading* 
module. Please see attachment, this is simplest python script +
c++ code which demonstrates problem.

By using threading.setprofiler I was able to make problem appear each run.
If you uncomment  time.sleep(1) problem won't reproduce.

In main.cpp I manually create separate interpreter because in real-life
application is multi-threaded and different thread uses different interpreters
for at least some sand-boxing.


I did some investigation, and here is what I found out:

* Each new thread is started by calling *thread_PyThread_start_new_thread* from threadmodule.c.
Basically this function creates new thread which executes *t_bootstrap* and adds this
thread to interpreter's thread list.

* *t_bootstrap* executes python function *Thread.__bootstrap* and when it's done, removes
thread from interpreter's thread list.

* *Thread.__bootstrap* runs *Thread.run()* (actually python code which should be executed
in separate thread) and when it's done calls *Thread.__stop* (which by mean of condition
variable sets Boolean flag Thread.__stopped)

* *Thread.join* doesn't wait thread to exit, it only waits for *Thread.__stopped* flag
to be set.

So here is race condition:
When main thread finished *Thread.join* it's only guaranteed that *Thread.run*
is finished, but *t_bootstrap* still may be running (interpreter's thread list
is not cleared).


To call Thread.__stop from *t_bootstrap* (instead of Thread.__bootstrap) after
removing thread from interpreter's thread list.

Or do not use detached threads and call something like *pthread_join* in *Thread.join*.
Which is IMHO much clearer approach but also requires much more efforts.

components: Extension Modules
files: main.cpp
messages: 246035
nosy: Oleksiy Markovets
priority: normal
severity: normal
status: open
title: Race condition and crash when embedding multi-thread script
type: crash
versions: Python 2.7
Added file: http://bugs.python.org/file39840/main.cpp

Python tracker <report at bugs.python.org>

More information about the New-bugs-announce mailing list