2008/9/26 Eljay Love-Jensen <eljay@adobe.com>
Hi everyone,
First, my apologies if I'm in the wrong forum for my "embedding Python in a C application" questions. Please redirect me if I've wandered into the wrong place.
I have two needs for using Python in my application that I hope has an easy answer without rewriting Python's internals.
I need to use Python* in a multi-threaded application, where separate threads may be working on very long lasting Python scripts, and other threads may be involved in short Python scripts. None of the Python scripts running concurrently have any shared state with any of the other Python scripts running concurrently. Number of threads is in the 100-1000 range.
I need to manage Python's use of the heap by providing a memory pool for Python to use, rather than allowing Python to use malloc/free. This is to prevent memory fragmentation, and to allow easy disposal of a memory pool used for a closed Python interpreter instance.
A quick view of Py_Initialize() indicates that Python does not return some sort of "Py_State" pointer which represents the entire state of a Python interpreter. (Nor some sort of Py_Alloc().) Nor accepts a custom malloc/free function pointers. Hmmm.
Python already has its own highly optimized memory allocator, it does not use malloc/free directly. That's why the configure option --without-pymalloc exists.
So I think your basic premise is wrong. But in any case maybe you are looking for PyInterpreterState_New(). But beware that going down that path is going to be painful: multiple interpreter states and threading can lead to many hours of debugging. I would think thrice before deciding I really need it.
Does anyone have experience with using Python in this fashion?
I remember trying to debug xchat python plugin interface which used multiple interpreter states and multiple threads. I wish I could forget those horrors...
(If relevant, it will be Python 3.x not Python 2.x.)
Thanks, --Eljay
- Doesn't HAVE to be Python. Could be JavaScript or Lua or whatnot. My preferences these days is a Python solution.
capi-sig mailing list capi-sig@python.org http://mail.python.org/mailman/listinfo/capi-sig
-- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert