On Fri, Sep 26, 2008 at 11:18 PM, Gustavo Carneiro <gjcarneiro@gmail.com>wrote:
2008/9/26 Eljay Love-Jensen <eljay@adobe.com>
Hi everyone,
First, my apologies if I'm in the wrong forum for my "embedding Python in a C application" questions. Please redirect me if I've wandered into the wrong place.
I have two needs for using Python in my application that I hope has an easy answer without rewriting Python's internals.
I need to use Python* in a multi-threaded application, where separate threads may be working on very long lasting Python scripts, and other threads may be involved in short Python scripts. None of the Python scripts running concurrently have any shared state with any of the other Python scripts running concurrently. Number of threads is in the 100-1000 range.
I need to manage Python's use of the heap by providing a memory pool for Python to use, rather than allowing Python to use malloc/free. This is to prevent memory fragmentation, and to allow easy disposal of a memory pool used for a closed Python interpreter instance.
A quick view of Py_Initialize() indicates that Python does not return some sort of "Py_State" pointer which represents the entire state of a Python interpreter. (Nor some sort of Py_Alloc().) Nor accepts a custom malloc/free function pointers. Hmmm.
Python already has its own highly optimized memory allocator, it does not use malloc/free directly. That's why the configure option --without-pymalloc exists.
So I think your basic premise is wrong. But in any case maybe you are looking for PyInterpreterState_New(). But beware that going down that path is going to be painful: multiple interpreter states and threading can lead to many hours of debugging. I would think thrice before deciding I really need it.
But If Eljay is trying to actually create multiple threads to run the scripts simultaneously, then I guess he has much more to worry about than just PyInterpreterState. The API does not take care of all the Python global variables for sure.
--Swapnil Talekar