I see. Thank you for the explanation. I use multi-phase initialisation now, but there is another problem. Last time, I found that Py_FinalizeEx() can't destroy all sub-interpreters, Victor told me that I must end each sub-interpreter manually ( https://bugs.python.org/issue36225). I tried to do it, like this (I leave out the function m_methods and m_slots below):
sem_t sem_start; sem_t sem_created; sem_t sem_end; pthread_t tid;
static void newinterFree(void *arg) { sem_post(&sem_start); printf("it's the free function\n"); Py_BEGIN_ALLOW_THREADS sem_wait(&sem_end); Py_END_ALLOW_THREADS }
static struct PyModuleDef newintermodule = { PyModuleDef_HEAD_INIT, .m_name = "newinter", .m_doc = NULL, .m_size = 0, .m_methods = NewinterMethods, .m_slots = mySlots, .m_traverse = NULL, .m_clear = NULL, .m_free = newinterFree };
void *thr_func(void *arg) { PyThreadState *thread_state, *new_thread_state;
PyGILState_STATE state = PyGILState_Ensure();
thread_state = PyGILState_GetThisThreadState();
new_thread_state = Py_NewInterpreter();
PyThreadState_Swap(thread_state);
PyGILState_Release(state);
PyThreadState_Swap(new_thread_state);
sem_post(&sem_created);
sem_wait(&sem_start);
state = PyGILState_Ensure();
Py_EndInterpreter(new_thread_state);
PyGILState_Release(state);
sem_post(&sem_end);
return ((void *)0);}
PyMODINIT_FUNC PyInit_newinter(void) { PyObject *m; m= PyModuleDef_Init(&newintermodule);
sem_init(&sem_created, 0, 0);
sem_init(&sem_start, 0, 0);
sem_init(&sem_end, 0, 0);
pthread_create(&tid, NULL, thr_func, NULL);
Py_BEGIN_ALLOW_THREADS
sem_wait(&sem_created);
Py_END_ALLOW_THREADS
return m;}
but the program is blocked when acquiring the GIL for Py_EndInterpreter, since at that time, the thread is already exiting. If I check that pthread_exit is called or not, I get the backtrace: #0 __pthread_exit (value=value@entry=0x0) at pthread_exit.c:25 #1 0x00005555556dcfc1 in PyThread_exit_thread () at ../Python/thread_pthread.h:361 #2 0x000055555567f072 in take_gil (tstate=tstate@entry=0x7ffff0000b30) at ../Python/ceval_gil.h:224 #3 0x000055555567f7f5 in PyEval_RestoreThread (tstate=tstate@entry=0x7ffff0000b30) at ../Python/ceval.c:467 #4 0x00005555556c9c15 in PyGILState_Ensure () at ../Python/pystate.c:1378
I saw the issue https://bugs.python.org/issue36479, it seems very close to my problem. I would like to ask how can I solve this problem? Can we set a thread to non-daemon thread?
Best, Kun
Nick Coghlan <ncoghlan@gmail.com> 于2021年4月17日周六 下午4:34写道：
On Wed, 14 Apr 2021, 9:03 am Kun He, <hekun19890913@gmail.com> wrote:
I'm afraid that I misunderstood your previous mail. When you said " it was basically not possible to unload a C extension.", it means that we can't delete the module from sys.modules (del sys.modules['_abc'])?
Extension modules that use the legacy initialisation API have a hidden copy of their top level namespace created.
If you delete them from sys.modules, reimporting them will create a new module object, but it won't recreate the *contents* of the module - those will be taken from the hidden copy rather than executing the module initialisation again.
Only extension modules that use the newer multi-phase initialisation API can be truly unloaded and reloaded.
Cheers, Nick.