[Python-Dev] thread issues when embedding Python

[Nick Coghlan]

On 19 December 2013 21:28, Daniel Pocock <daniel at pocock.com.au> wrote:
> On 19/12/13 12:22, Nick Coghlan wrote:
>> I don't see anything in your article about how you ensure that the
>> main thread of the application *before anything else related to the
>> embedded Python happens* calls both Py_Initialize() and
>> PyEval_InitThreads(). If you don't do that, then all bets are off in
>> terms of multithreading support.
>
> I definitely do both of those things in the method PyRoutePlugin::init(..)
>
> It is in PyRoutePlugin.cxx:
>
> http://svn.resiprocate.org/viewsvn/resiprocate/main/repro/plugins/pyroute/PyRoutePlugin.cxx?view=markup#l88

I can't see an immediately obvious explanation for why your current
approach based on PyExternalUser::Use gets things working, while the
PyThreadSupport approach fails immediately. However, you'd need to be
able to reproduce the problem with a much simpler embedding
application and without PyCXX involved anywhere to confirm it as a
possible CPython bug, or to identify exactly what is missing from the
current embedding initialisation instructions.

The reason for that is the fact that the GIL state API is unit tested
on a wide variety of platforms inside a fully initialised interpreter
and that means we know this *does* work in the absence of external
interference: http://hg.python.org/cpython/file/16bfddf5a091/Modules/_testcapimodule.c#l1335

I also asked Graham Dumpleton (author of mod_wsgi, one of the more
complex CPython embedding scenarios currently in existence) to take a
look, and he didn't see any obvious explanation for the discrepancy
either, so you may want to try a cut down implementation without PyCXX
to see if that's the culprit.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia