On Tue, 22 Jan 2019 15:32:22 +0100
Stephan Reiter
Numpy's extension umath calls PyGILState_Ensure(), which in turn calls PyEval_RestoreThread on the (auto) threadstate of the main interpreter. And that's wrong. We are already holding the GIL with the threadstate of our current sub-interpreter, so there's no need to switch.
I know that the GIL API is not fully compatible with sub-interpreters, as issues #10915 and #15751 illustrate.
That's a pity. Note that there is a patch on https://bugs.python.org/issue10915 that could probably solve the issue if it had been applied some years ago ;-) (yes, it needs C extension authors to use the new API, but Numpy is a well-maintained library and would probably have accepted a patch for that; so would Cython probably)
Essentially, I'd like to ask for some guidance in how to tackle this problem while keeping the current GIL API unchanged (to avoid breaking modules).
I'm not aware of any solution which does not require designing a new API, unfortunately.
I am also wondering how I can test any changes I am proposing. Is there a test suite for interpreters, for example?
You'll find a couple of them in test_embed.py, test_capi.py and test_threading.py. Regards Antoine.