[capi-sig] Embedded Python in C application

[Gustavo Carneiro]

2008/9/26 Eljay Love-Jensen <eljay at adobe.com>

> Hi everyone,
>
> First, my apologies if I'm in the wrong forum for my "embedding Python in a
> C application" questions.  Please redirect me if I've wandered into the
> wrong place.
>
> I have two needs for using Python in my application that I hope has an easy
> answer without rewriting Python's internals.
>
> I need to use Python* in a multi-threaded application, where separate
> threads may be working on very long lasting Python scripts, and other
> threads may be involved in short Python scripts.  None of the Python
> scripts
> running concurrently have any shared state with any of the other Python
> scripts running concurrently.  Number of threads is in the 100-1000 range.
>
> I need to manage Python's use of the heap by providing a memory pool for
> Python to use, rather than allowing Python to use malloc/free.  This is to
> prevent memory fragmentation, and to allow easy disposal of a memory pool
> used for a closed Python interpreter instance.
>
> A quick view of Py_Initialize() indicates that Python does not return some
> sort of "Py_State" pointer which represents the entire state of a Python
> interpreter.  (Nor some sort of Py_Alloc().)  Nor accepts a custom
> malloc/free function pointers.  Hmmm.


Python already has its own highly optimized memory allocator, it does not
use malloc/free directly.  That's why the configure option
--without-pymalloc exists.

So I think your basic premise is wrong.  But in any case maybe you are
looking for PyInterpreterState_New().  But beware that going down that path
is going to be painful: multiple interpreter states and threading can lead
to many hours of debugging.  I would think thrice before deciding I really
need it.


> Does anyone have experience with using Python in this fashion?


I remember trying to debug xchat python plugin interface which used multiple
interpreter states and multiple threads.  I wish I could forget those
horrors...


>
>
> (If relevant, it will be Python 3.x not Python 2.x.)
>
> Thanks,
> --Eljay
>
> * Doesn't HAVE to be Python.  Could be JavaScript or Lua or whatnot.  My
> preferences these days is a Python solution.
>
> _______________________________________________
> capi-sig mailing list
> capi-sig at python.org
> http://mail.python.org/mailman/listinfo/capi-sig
>



-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert