xmlrpc idea for getting around the GIL
fetchinson at googlemail.com
Tue Nov 24 12:15:39 CET 2009
>> icating) the multiprocessing module would be ideal.
>>> > The problem is that the OP has a embedded application running threads.
>>> > multiprocssing doesn't help there.
>>> that's right. I cannot make CPython calls from my original C-based
>> It's quite possible to do that. A thread started from C can make
>> calls to Python if it first calls PyGILState_Ensure, although you'd
>> have to make sure that the Python interpreter has been previously
>> initialized. See PEP 311 for details.
>> I also suggest that if you want people to be more receptive to write
>> your replies after the quoted text. That is the custom in this
>> newsgroup/mailing list.
> What I meant was that I am *not allowed* to make calls to the CPython
> API from the threads I currently have because these threads are high
> priority and are never allowed to make blocking calls. Fortunately,
> each of these threads can have a completely separate interpreter, so
> my idea was to create a daemon process for each thread. This daemon
> would contain the libpython symbols and would make the CPython calls.
> This would keep my current threads from having to contend over the
> single GIL.
> My question was whether or not anyone has done anything like this in
> C/C++. This situation is slightly unique in that I am trying to
> maintain separate interpreters within a single app (which is currently
> kind of hacked out using a single interpreter, but it's ugly), but I
> could see how this sort of thing would be useful for other C/C++ apps
> that implement an embedded scripting engine.
> My reference to multiprocessing was based on the idea that the library
> hides the details fo the process management, shared memory, and rpc
> mechanisms. Again, I can't use multiprocessing because it runs *in*
> python I need this to be implemented *outside* of python to avoid
> acquiring the GIL. complex, I know.
> Naturally, the most intimidating part of perusing this kind of idea is
> the overhead of the processess management and the RPC. Further,
> libpython executes callable objects using C function pointers, and I
> can't think of a way that this would be able to re-gain access to the
> original app's respective functions if the interpreter was living in
> another processes.
> That's not to mention the impossibility of debugging.
> Damn you, Gil.
By the way, you might be interested in this thread as well:
Psss, psss, put it down! - http://www.cafepress.com/putitdown
More information about the Python-list