There are currently numerous incompatibilities between numpy and
subinterpreters, and no concrete plan for fixing them. The numpy team does
not consider subinterpreters to be a supported configuration, and can't
help you with any issues you run into. I know the concept of
subinterpreters is really appealing, but unfortunately the CPython
implementation is not really mature or widely supported... are you
absolutely certain you need to use subinterpreters for your application?
On Tue, Jan 22, 2019, 08:27 Stephan Reiter Hi all! I am new to the list and arriving with a concrete problem that I'd
like to fix myself. I am embedding Python (3.6) into my C++ application and I would like
to run Python scripts isolated from each other using sub-interpreters.
I am not using threads; everything is supposed to run in the
application's main thread. I noticed that if I create an interpreter, switch to it and execute
code that imports numpy (1.13), my application will hang. python36.dll!_PyCOND_WAIT_MS(_PyCOND_T * cv=0x00000000748a67a0,
_RTL_CRITICAL_SECTION * cs=0x00000000748a6778, unsigned long ms=5) Line 245
C
[Inline Frame] python36.dll!PyCOND_TIMEDWAIT(_PyCOND_T *) Line 275 C ntdll.dll!NtWaitForSingleObject() Unknown
KernelBase.dll!WaitForSingleObjectEx() Unknown
python36.dll!take_gil(_ts * tstate=0x0000023251cbc260) Line 224 C
python36.dll!PyEval_RestoreThread(_ts * tstate=0x0000023251cbc260) Line
370 C
python36.dll!PyGILState_Ensure() Line 855 C
umath.cp36-win_amd64.pyd!00007ff8c6306ab2() Unknown
umath.cp36-win_amd64.pyd!00007ff8c630723c() Unknown
umath.cp36-win_amd64.pyd!00007ff8c6303a1d() Unknown
umath.cp36-win_amd64.pyd!00007ff8c63077c0() Unknown
umath.cp36-win_amd64.pyd!00007ff8c62ff926() Unknown
[Inline Frame] python36.dll!_PyObject_FastCallDict(_object *) Line 2316 C
[Inline Frame] python36.dll!_PyObject_FastCallKeywords(_object *) Line
2480 C
python36.dll!call_function(_object * * *
pp_stack=0x00000048be5f5e40, __int64 oparg, _object * kwnames) Line
4822 C Numpy's extension umath calls PyGILState_Ensure(), which in turn calls
PyEval_RestoreThread on the (auto) threadstate of the main
interpreter. And that's wrong.
We are already holding the GIL with the threadstate of our current
sub-interpreter, so there's no need to switch. I know that the GIL API is not fully compatible with sub-interpreters,
as issues #10915 and #15751 illustrate. But since I need to support calls to PyGILState_Ensure - numpy is the
best example -, I am trying to improve the situation here: https://github.com/stephanreiter/cpython/commit/d9d3451b038af2820f500843b6a8... That change may be naive, but it does the trick for my use case. If
totally wrong, I don't mind pursuing another alley. Essentially, I'd like to ask for some guidance in how to tackle this
problem while keeping the current GIL API unchanged (to avoid breaking
modules). I am also wondering how I can test any changes I am proposing. Is
there a test suite for interpreters, for example? Thank you very much,
Stephan
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/njs%40pobox.com