C integration with multithreaded Python script

Hope this is the right mailing list, please complain if not ;)
I'm currently experimenting with the integration of Python (2.6) scripts into my application, and am using the following script for testing:
*import* threading, time
*def* handler():
print("a")
time.sleep(0.95)
print("b")
t = threading.Thread(target = handler)
t.start()
And in C++, it's this code:
Py_SetProgramName("test");
Py_Initialize();
*if*(!Py_IsInitialized()) *throw* std::exception();
PyObject *module = PyImport_AddModule("__main__");
PyObject *dict = PyModule_GetDict(module);
printf("1\n");
strcpy(script, "import threading, time\n"
"def handler():\n"
" print(\"a\")\n"
" time.sleep(0.95)\n"
" print(\"b\")\n"
"t = threading.Thread(target = handler)\n"
"t.start()\n");
PyObject *obj = PyRun_String(script, Py_file_input, dict, dict);
printf("2\n");
PyErr_Print();
Py_Finalize();
When executing, the output is
1 a2
as opposed to the expected
1 a b 2
It seems to me that when the main execution path (after t.start()) finishes, the other thread (called t) is canceled?! Adding t.join() to the end of the script gives the expected output, but isn't it possible to let PyRun_xxx wait for all remaining threads to end? I know the Python interpreter does this, but I didn't get much information from its source code (it seems to run scripts with the runpy module).
Best regards

I still don't have an answer to this question. Meanwhile I've taken a look at the Python interpreter source code and couldn't find out how it manages to keep running while other threads exist. Taking the /RunModule/ function from the code doesn't help me - same problem: only the first few instructions of the background thread are executed and then Python execution ends.
Andreas Sommer wrote:
Hope this is the right mailing list, please complain if not ;)
I'm currently experimenting with the integration of Python (2.6) scripts into my application, and am using the following script for testing:
*import* threading, time
*def* handler(): print("a") time.sleep(0.95) print("b")
t = threading.Thread(target = handler) t.start()
And in C++, it's this code:
Py_SetProgramName("test"); Py_Initialize(); *if*(!Py_IsInitialized()) *throw* std::exception();
PyObject *module = PyImport_AddModule("__main__"); PyObject *dict = PyModule_GetDict(module);
printf("1\n");
strcpy(script, "import threading, time\n" "def handler():\n" " print(\"a\")\n" " time.sleep(0.95)\n" " print(\"b\")\n" "t = threading.Thread(target = handler)\n" "t.start()\n");
PyObject *obj = PyRun_String(script, Py_file_input, dict, dict); printf("2\n");
PyErr_Print(); Py_Finalize();
When executing, the output is
1 a2
as opposed to the expected
1 a b 2
It seems to me that when the main execution path (after t.start()) finishes, the other thread (called t) is canceled?! Adding t.join() to the end of the script gives the expected output, but isn't it possible to let PyRun_xxx wait for all remaining threads to end? I know the Python interpreter does this, but I didn't get much information from its source code (it seems to run scripts with the runpy module).
Best regards

Okay, I found out that the function WaitForThreadShutdown in modules/main.c is responsible for calling threading._shutdown which will automatically join all non-daemonic threads.
Hope this will help somebody else...
Andreas Sommer wrote:
I still don't have an answer to this question. Meanwhile I've taken a look at the Python interpreter source code and couldn't find out how it manages to keep running while other threads exist. Taking the /RunModule/ function from the code doesn't help me - same problem: only the first few instructions of the background thread are executed and then Python execution ends.
Andreas Sommer wrote:
Hope this is the right mailing list, please complain if not ;)
I'm currently experimenting with the integration of Python (2.6) scripts into my application, and am using the following script for testing:
*import* threading, time
*def* handler(): print("a") time.sleep(0.95) print("b")
t = threading.Thread(target = handler) t.start()
And in C++, it's this code:
Py_SetProgramName("test"); Py_Initialize(); *if*(!Py_IsInitialized()) *throw* std::exception();
PyObject *module = PyImport_AddModule("__main__"); PyObject *dict = PyModule_GetDict(module);
printf("1\n");
strcpy(script, "import threading, time\n" "def handler():\n" " print(\"a\")\n" " time.sleep(0.95)\n" " print(\"b\")\n" "t = threading.Thread(target = handler)\n" "t.start()\n");
PyObject *obj = PyRun_String(script, Py_file_input, dict, dict); printf("2\n");
PyErr_Print(); Py_Finalize();
When executing, the output is
1 a2
as opposed to the expected
1 a b 2
It seems to me that when the main execution path (after t.start()) finishes, the other thread (called t) is canceled?! Adding t.join() to the end of the script gives the expected output, but isn't it possible to let PyRun_xxx wait for all remaining threads to end? I know the Python interpreter does this, but I didn't get much information from its source code (it seems to run scripts with the runpy module).
Best regards
capi-sig mailing list capi-sig@python.org http://mail.python.org/mailman/listinfo/capi-sig

Andreas Sommer wrote:
Okay, I found out that the function WaitForThreadShutdown in modules/main.c is responsible for calling threading._shutdown which will automatically join all non-daemonic threads.
Hope this will help somebody else...
Thank you for posting that follow up to the issue.
-Jeff
participants (2)
-
Andreas Sommer
-
Jeff Rush