[Python-Dev] thread issues when embedding Python

Daniel Pocock daniel at pocock.com.au
Wed Dec 18 00:07:17 CET 2013

I've successfully embedded Python for a single thread

I tried to extend the implementation for multiple threads (a worker
thread scenario) and I'm encountering either deadlocks or seg faults
depending upon how I got about it.

There seems to be some inconsistency between what is covered in the docs


and the experiences of other users trying the same thing, e.g.


Can anybody comment on the situation, in particular,

Is the non-python-created-threads documentation accurate for v2.7?

If a main thread does things like importing a module and obtaining a
reference to a Python method, can those things be used by other C++
threads or do they have to repeat those lookups?

Is there any logic that needs to be executed once only as each thread is
started? (the doc suggests just PyGILState_Ensure/PyGILState_Release
each time a thread accesses Python methods - is there anything else?)

Given the bug 19576, what is the most portable way to code this to work
reliably on unfixed Python versions?  (e.g. should users always
explicitly call PyEval_InitThreads() in their main thread or worker
threads or both?)

Here is my actual source code:


  (see example.py for a trivial example of what it does)

The problem that I encounter:

- the init stuff runs fine in PyRoutePlugin.cxx,
    it calls Py_Initialize, PyEval_InitThreads, PyImport_ImportModule
      and looks up the "provide_route" method in the module
    it creates a PyRouteWorker object,
      giving it a reference to "provide_route"
    it creates a thread pool to run the worker

- the PyRouteWorker::process() method is invoked in one of those threads

- it crashes when trying to call the "provide_route" method
      routes = mAction.apply(args);

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff30b8700 (LWP 23965)]
0x00007ffff3d6ad06 in PyObject_Call () from /usr/lib/libpython2.7.so.1.0
(gdb) bt
#0  0x00007ffff3d6ad06 in PyObject_Call () from /usr/lib/libpython2.7.so.1.0
#1  0x00007ffff3d6b647 in PyEval_CallObjectWithKeywords ()
   from /usr/lib/libpython2.7.so.1.0
#2  0x00007ffff414885a in apply (args=..., this=<optimized out>)
    at /usr/include/python2.7/CXX/Python2/Objects.hxx:3215
#3  repro::PyRouteWorker::process (this=0x6f00a0, msg=<optimized out>)
    at PyRouteWorker.cxx:98
#4  0x00007ffff7b879e1 in repro::WorkerThread::thread (this=0x68e110)
    at WorkerThread.cxx:36
#5  0x00007ffff70b7a2f in threadIfThreadWrapper (threadParm=<optimized out>)
    at ThreadIf.cxx:51
#6  0x00007ffff65ffb50 in start_thread (arg=<optimized out>)
    at pthread_create.c:304
#7  0x00007ffff5999a7d in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#8  0x0000000000000000 in ?? ()

More information about the Python-Dev mailing list