Question about Python threads

[Mr. Neutron]

On Wed, 21 Aug 2002 23:28:09 -0400, Aahz wrote:

> In article <pan.2002.08.21.14.18.37.335853.19857 at charter.net>, Mr.
> Neutron <nicktsocanos at charter.net> wrote:
>>
>>  If I understand, only one thread can be in the interpreter at any
>>  time.
>>Now what I don't understand is being in the interpreter at a time part.
> 
> That means pure Python code.  For example:
> 
>     x = 1 + 2
> 
> However, if you call a C function that releases the global interpreter
> lock (GIL), then multiple threads can run concurrently.  Most I/O
> functions in Python do this.  For example:
> 
>     page = urllib.urlopen('http://www.python.org/').read()

Ok entertain a newbie to Python here for a moment. I am not new to C/C++
though, so I have confidence I could use C to interface to Python.

Now suppose i want to write a thread in python and I want it to execute
on two CPUs. Ok, I have two threads, I will call them CPUA() and CPUB(). 

Ok first, a few questions about C development with threads on Linux.

First, imagine CPUA() and CPUB() are now C functions. How do I specifically
tell in C using pthreads, that I
want CPUA() on CPU0 and CPUB() on CPU1? My understanding is you can't
specify this on Linux, but I hope I am just limited in my understanding
of pthreads.Not a python question, but it is related to something I will
ask you in a few seconds below..

Second, how in C do I access the GIL in python? Where do I look for this
or are there any examples of this?

Third, where to look in Python source tree for the threading mechanisms?
I am studying Python source to learn. I will eventually find it on my
own, but any experience knowing where to look will speed things up. I
want to see how it works not that I have much intentions to change
anything at this point. Purely academic reasons. I am interested in
interpreted language design. I also study Blackdown JVM too.

Now "the one thread in the interpreter problem" on a single CPU computer is
fine anways, as only one thread will be executing in a slice of time
anyways, so it is really no big deal right. I am comfortable with that.

*Now the real problem is when you have more than one CPU. Now you in
theory have two threads running at the same exact time. *

I can think ways to get around this. One, I can make forks of my
program right? Just write my program to behave like a fork. But that stil
leaves me wondering if fork creates an entirely new Python interpreter or
not, and whether it will get scheduled on both CPUs as well.

Two I can write a C program which takes two Python interpreters and puts
one on each CPU. Then I can have concurrent Pythons running and
everything is ok right? Then I can have CPUA() on Python0 and CPUB() on
Python1, and they are not threads, but literal Python programs. Then they
are running in C as threads on both CPUS. This seems like a solution to
running Python on more CPUs but is not a very satisfactory result i want.
I would have to make IPC for both programs to communicate etc. But I
suppose this can be done. But really for what I want to do, this is fine,
as my threads communicate through a socket. So it is ok.

Three, I can write my system performance code in C. This will deal with
making the threaded processes I need on both CPUs. I wrap it in a Python
program that communicates to these threads. Again I need a pipe or
something to talk between processes. This is not what I want. I want to
do my coding in as much Python as possible. If I want C, I would just
write in C in the first place. The purpose is to push my understanding of
Python.

Four, I figure out how to release the GIL.

So really, if I want to specifically write a program for multiple CPUs
(SMP) with Python, what do I need to research into and where can I look
for examples? How do I use GIL, and where do I find it? I have thought of
some simple ways to get around the thread problem on multiple CPUs. Are
there any better ways than what I have thought of?

I would much rather use Python, a language I find easy to learn and
program in, than learn an obscure language like Cilk. Not bashing Cilk,
it looks wonderful. But I am lazy and don't feel like learning a new
language if I don't have to.

Thanks