Python, Embedding Q's

B Kingsford bryn at
Fri Aug 6 07:23:44 CEST 1999

hi all

im embedding Python in multi-threaded server, and just had a
couple of q's regarding compiling files, cache, etc. if any help
or thoughts could be offered, please do.

I wish to use Python's built-in threading of the interpreter,
aswell as Python's memory management [for portability], will
this have any major, adverse performance effects?
Does Python handle which thread does what?
would it be better to create say, x threads, each with their
own instance of the interpreter?

what is the PyDict format of globals/locals from PyModule_GetDict?
what vars are declared in this ModuleDict? is it necessary to add
anything there?

does PyCodeObject keep (or represent) the compiled code in mem?
would it be worth maintaining a heap of PyCodeObjects (one for
each .pyx, or until cache is full...) as opposed to recompiling
for each request (unless it is desired, say, .pyd), say
PyCodeObject  * pcoCached[] = malloc( 500000 );
will Py use this new heap for the CodeObject pointers aswell as?
will the CodeObjects fill the heap, or will Py, when compiling
for a CodeObject, create mem elsewhere? eer, does sizeof(PyCodeObject)
include the Py bytecode?

if i were to maintain the PyCodeObjects, how then would i have
an instance of the interpreter run this code? ahh, obviously
PyEval_EvalCode, but what if Python is handling the sub-interpreters?
how to say, ThatSubInterpreter->EvalCode( PyCodeObject )?

is Py_NewInterpreter really a 'fresh copy' of the main interpreter
for use by a thread? if i add built-in mods with PyImport_AddModule,
do i need to do this for each Py_NewInterpreter? what about
Py_InitModule? does this need to be called for each new aswell?

in the examples, the telnet server seems to call Py_NewInterpreter
for each request. would i have to, if say i wished to limit the
worker threads, when a request occurs, if threads >= max, put
request in queue, OR, create a new thread with Py_NewInterpreter,
handle the request, check to see if any new requests are in the
queue [and handle them too], then Py_EndInterpreter? what about
reusing Py_NewInterpreters after they've been idle for a while?
(say, a LIFO q of interpreters... you know how it is... no action
for a while then bang, everybody wants in on your server!!)

cheers and thanks in advance

More information about the Python-list mailing list