Threading Pool Event()

[Alan Kennedy]

Graeme Matthew wrote:

> here is my code, please criticise it to death i wont take offence

[snip]

> #ON A MULTIPLE CPU MACHINE 2 SEPERATE SERVERS SHOULD BE RUN
> #THE INCOMING REQUEST WILL NEED TO BE SWITCHED BETWEEN SERVERS
> #THIS IS DUE TO THE GLOBAL INTERPRETER LOCK (GIL) IN PYTHON

If my understanding of your intentions is correct, then I think you're
expecting that multiple python threads within the same interpreter
will migrate to multiple processors, and thus execute in parallel,
i.e. simultaneously on multiple processors.

However, the GIL prevents precisely that: if you have one python
thread running on one processor, it will lock the GIL, thus causing
all other python threads on other processors to be suspended until the
GIL is free again.

One way to achieve true concurrency across multiple processors is
to release the GIL when you are processing, which you cannot do from
within python code. You have to write a C language extension to do the
work, that releases the GIL before doing its job, and reacquires it
again when finished.

If you really want true concurrency, and still want to write in pure
python, then one way to do it is run multiple python interpreters in
separate OS processes, which are bound to different processors. Which
means that

1. You can no longer use Queue.Queue objects to communicate between
servers, since your servers no longer share an address space.

2. You have to use some alternate comms mechanism between the two
servers, such as shared memory, named pipes or something like Pyro.

3. You can no longer pass the socket object across wrapped in a "job"
object, since the file descriptor tables of the two server processes
will be different.

Lastly, I have a question of my own: I've read statements along the
lines of "having the GIL in python provides certain guarantees that
make certain things possible". I can't remember specifics. I'd be most
grateful if someone could point me to reading material (apart from the
source, Luke :-) which describes what these guarantees are, and what
they make possible.

TIA,

--
alan kennedy
-----------------------------------------------------
check http headers here: http://xhaus.com/headers
email alan:              http://xhaus.com/mailto/alan