As I see in the document, this function can destroy all sub-interpreters
Sadly, I don't think that it's the case. You must end each sub-interpreter manually. https://bugs.python.org/issue36225
Victor
On Mon, Apr 12, 2021 at 12:38 PM Kun He <hekun19890913@gmail.com> wrote:
Hi Victor,
I added PyGILState_Ensure()/PyGILState_Release() when calling Py_NewInterpreter(), new interpreters can be created.
But during the finalizing state, Py_FinalizeEx() is called by the program itself. As I see in the document, this function can destroy all sub-interpreters, but it is not the case in my code. There is an error "Fatal Python error: PyInterpreterState_Delete: remaining subinterpreters". I don't know why it happens?
I suppose that I can call Py_EndInterpreter() manually before Py_FinalizeEx() is called, but I don't know which function is used for finalizing (is there a function like PyInit_modulename but reverse)?
Best regards, Kun
Victor Stinner <vstinner@python.org> 于2021年4月12日周一 上午10:17写道：
You need to call PyGILState_Ensure()/PyGILState_Release() around the the call to the C extension function. These function creates a Python thread state to the newly created thread.
So when calling Py_NewInterpreter() I don't have the GIL.
Sorry, but this is wrong. All functions of the C API require to hold the GIL. If it's not the case, it's a bug in your code.
Victor
On Fri, Apr 9, 2021 at 6:06 PM Kun He <hekun19890913@gmail.com> wrote:
Hi Victor,
Thank you for handling my problem.
In fact, in my actually processed C interface, I need to call a C extension function which calls Py_NewInterpreter() in new thread, and I put Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS around this function. So when calling Py_NewInterpreter() I don't have the GIL.
Best regards, Kun
Victor Stinner <vstinner@python.org> 于2021年4月9日周五 下午3:57写道：
Aha, I reproduce your issue: https://bugs.python.org/issue43793
Currently, I understand that you must first acquire the GIL using PyGILState_Ensure()/PyGILState_Release() in the newly created thread to be able to call Py_NewInterpreter(). See: https://docs.python.org/dev/c-api/init.html#non-python-created-threads
Maybe tomorrow it will be possible to call Py_NewInterpreter() on a newly created thread which has no thread state.
Victor
On Fri, Apr 9, 2021 at 2:26 PM Kun He <hekun19890913@gmail.com> wrote:
Hi Victor,
Thanks for your response. I'm sorry that the attachment is not available. Anyway, I copy the C program as you can see below:
#include <stdio.h> #include <string.h>
#include <pthread.h>
#define PY_SSIZE_T_CLEAN #include <Python.h>
static PyObject* print_str(PyObject* self, PyObject* args) { char* content;
if (!PyArg_ParseTuple(args, "s", &content)) return NULL; printf("we want to print %s\n", content); Py_INCREF(Py_None); return Py_None;}
static PyMethodDef NewinterMethods[] = { {"print_str", print_str, METH_VARARGS, "print the input string"}, {NULL, NULL, 0, NULL} };
static struct PyModuleDef newintermodule = { PyModuleDef_HEAD_INIT, "newinter", NULL, -1, NewinterMethods };
pthread_t tid;
void *thr_func(void *arg) { Py_NewInterpreter(); return ((void *)0); }
PyMODINIT_FUNC PyInit_newinter(void) { PyObject *m;
m = PyModule_Create(&newintermodule);#if PY_MAJOR_VERSION < 3 || (PY_MAJOR_VERSION == 3 && PY_MINOR_VERSION < 9) PyEval_InitThreads(); #endif
int err = pthread_create(&tid, NULL, thr_func, NULL); if(err != 0) { printf("can't create thread\n"); return NULL; } else printf("create new thread\n"); return m;}
And for test, a very simple Python script is used as below: import newinter newinter.print_str("test")
Sincerely, Kun
Victor Stinner <vstinner@python.org> 于2021年4月9日周五 下午2:01写道：
Hi Kun,
Can you please try to provide a short C program reproducing your issue? Sadly, I cannot see your attachement. Try to copy it directly in the email body.
Victor
On Fri, Apr 9, 2021 at 12:09 PM Kun He <hekun19890913@gmail.com> wrote: > > Hello, > > I'm working on a C extension interface of Python. I want to create a new > interpreter by using the function Py_NewInterpreter() in a new thread, > which is created by pthread_create (my test files are in attachment), but > there are always errors when calling Py_NewInterpreter() such as "failed: > object already tracked by the garbage collector". > > I would like to ask how to solve the problem and create a new interpreter > in multi thread in Python C extension? > > Sincerely, > Kun > _______________________________________________ > capi-sig mailing list -- capi-sig@python.org > To unsubscribe send an email to capi-sig-leave@python.org > https://mail.python.org/mailman3/lists/capi-sig.python.org/ > Member address: vstinner@python.org
-- Night gathers, and now my watch begins. It shall not end until my death.
-- Night gathers, and now my watch begins. It shall not end until my death.
-- Night gathers, and now my watch begins. It shall not end until my death.
-- Night gathers, and now my watch begins. It shall not end until my death.