basic thread question

[sturlamolden]

On 18 Aug, 22:10, Derek Martin <c... at pizzashack.org> wrote:

> I have some simple threaded code...  If I run this
> with an arg of 1 (start one thread), it pegs one cpu, as I would
> expect.  If I run it with an arg of 2 (start 2 threads), it uses both
> CPUs, but utilization of both is less than 50%.  Can anyone explain
> why?  

Access to the Python interpreter is serialized by the global
interpreter lock (GIL). You created two threads the OS could schedule,
but they competed for access to the Python interpreter. If you want to
utilize more than one CPU, you have to release the GIL or use multiple
processes instead (os.fork since you are using Linux).

This is how the GIL can be released:

* Many functions in Python's standard library, particularly all
blocking i/o functions, release the GIL. This covers the by far most
common use of threads.

* In C or C++ extensions, use the macros Py_BEGIN_ALLOW_THREADS and
Py_END_ALLOW_THREADS.

* With ctypes, functions called from a cdll release the GIL, whereas
functions called from a pydll do not.

* In f2py, declaring a Fortran function threadsafe in a .pyf file or
cf2py comment releases the GIL.

* In Cython or Pyrex extensions, use a "with nogil:" block to execute
code without holding the GIL.


Sturla Molden