threading issue on Solaris SMP

Scott Gilbert xscottgjunk at yahoo.com
Wed Oct 16 07:42:11 CEST 2002


awalshe at iel.ie (Andy) wrote:
>
> Hello,
> I am using Solaris 8 and Python 2.2.1. We have a multi-threaded c++
> application which embeds python. The embedded python then loads python
> scripts. Python is built as a shared library.
> 
> When testing i send in multiple simultaneous requests which spawn a
> new thread. Each new thread then grabs a sub-interpreter from a pool,
> loads the relevant script and runs it.
> 

There really doesn't seem to be a way to let one process have multiple
interpreters running simultaneously.  You can get some concurrency by
having C extensions which release the Global Interpreter Lock (GIL),
but multiple pieces of Python script aren't going to execute
concurrently inside of one process.  I'm not sure what value there is
in creating multiple interpreters since you still only get to execute
one at a time.

That said, it also takes a good deal of work to have multiple C/C++
threads share a single embedded interpreter equally.  It's much easier
if you just have one thread that owns the interpreter, and have
everyone go through that thread for their Python needs.  This wasn't
sufficient for what we wanted to do at work.  We had to have multiple
threads trying to jump into the interpreter whenever they saw fit. 
Your situation sounds similar.

I wrote 4 routines: pv_init(), pv_acquire(), pv_release(), and
pv_term().  The pv_init() routine should be called before any threads
try to touch the interpreter (probably just inside of main() or a
global constructor).  Then each thread requests usage of the
interpreter by calling pv_acquire() and releases it by calling
pv_release().  So _all_ Python routines have to be called from between
the call to pv_acquire() and the call to pv_release() in order for
this to work.  Calling pv_term() is optional at program shutdown
depending on whether you want to cleanup gracefully or not...

    http://www.geocities.com/xscottg/pv_stuff.c

I ripped a bunch of stuff out that wasn't relevant to you, and I hope
I didn't break it in the process.  If you find it useful, I'd love to
hear about it (my real email address doesn't contain "junk").  If you
use it and find any bugs, I'd like to hear about that too.  :-)

Cheers,
    -Scott



More information about the Python-list mailing list