I am still confused about this. Maybe shorter, we have:
User who release GIL → NumPy API → third-party C function
We can probably assume that the user will use
NumPy API from random
threads, but only from threads already running Python (i.e. created by
Python), although there may be arguments that exceptions would be nice,
I do not think those are a priority.
In that case, is
PyGILState_Ensure() considered safe and appropriate
API to be used within "third-party C function" (and/or NumPy API)? I
used to generally think its not, but now it seems to me it should be,
if the thread we are running on was previously spawned by Python?
Or should it be a priority that e.g. the
PyThreadState *tstate is in
some form passed all the way down to the "third-part C function" so
that warnings or errors could be set more easily?
We could always try to design an API where the error setting is deferred until a time when the GIL is guaranteed to be held, but that seems inconvenient to me (it requires to pass out all error-information in some different way, only to convert it to an error later).
On Fri, 2020-08-07 at 12:04 -0500, Sebastian Berg wrote:
I am a bit confused about future proof GIL release with respect to potential subinterpreter development. Or maybe just the best practices.
In short, is the
PyGILState_Ensure()function safe? I had the impression that was definitely not, but now makes me wondered how even
PyErr_Occurred()can be safe when
EXPERIMENTAL_ISOLATED_SUBINTERPRETERS. Of course
PyErr_Occurred()is called with the GIL, so that may make a big difference.
I scanned through this thread: https://bugs.python.org/issue15751 which suggested the addition of API to make it (almost?) always safe? But most of that thread is very old, much older than the newer
EXPERIMENTAL_ISOLATED_SUBINTERPRETERSwork. Reading https://bugs.python.org/issue4202 makes me think there is no current solution/safety?
In my case (NumPy), I may be able to assume that the
PyGILState_Ensure()calls I am talking about will always happen in the same thread as the initial GIL release, which gives me some hope that it might actually just work? 
My impression until now was that the only clean solution would be to pass the threadstate. But, there is some public API, where we would require new API to pass the threadstate (which means slow adoption at the very least). So if I can avoid it, that would be much easier.
The consumer (the person who needs to grab the GIL) can be different from the user (the person releasing the GIL). So I am considering passing
PyThreadState **with NULL indicating "unknown" state to the consumer. Much code may not even release the GIL, and in either case a move to a new API has to take at least 1-2 years, so I can't really wait for that to be done.
 To be honest, I am not quite sure on this. It is true for NumPy itself, but I am not sure it is true for potential consumers of NumPy API. Or maybe, it would just be something to document very clearly somewhere. _______________________________________________ capi-sig mailing list -- email@example.com To unsubscribe send an email to firstname.lastname@example.org https://mail.python.org/mailman3/lists/capi-sig.python.org/ Member address: email@example.com