when does the GIL really block?
callen314 at gmail.com
Fri Aug 1 03:27:36 CEST 2008
I have followed the GIL debate in python for some time. I don't want
to get into the regular debate about if it should be gotten rid of
(though I am curious about the status of that for Python 3)...
personally I think I can do multi-threaded programming well, but I
also see the benefits of a multiprocess approach. I'm not so
egotistical that I don't realize perhaps my mt programming has not
been "right" (though it worked and was debuggable) or more likely that
doing it right I have avoided even trying some things people want mt
programming to do... i.e. to do mt programming right you start to use
queues a lot, inter-thread asynchronous, non-blocking, communication,
which is essentially the multi-process approach, using IPC (except
that the threads can see the same memory when, in your special case,
you know that's ok. Given something like a reader-writer lock, this
can have benefits... but again, whatever.
My question is that given this problem, years ago before I started
writing in python I wrote some short programs in python which could,
in fact, busy both my CPUs. In retrospect I assume I did not have
code in my run function that causes a GIL lock... so I have done this
I start two threads... I use gkrellm to watch my processors (dual
processor machine). If I merely print a number... both CPUS are
getting 90% simultaneous loads. If I increment a counter and print it
too, the same, and if I create a small list and sort it, the same. I
did not expect this... I expected to see one processor pegged at
around 100%, which should sometimes switch to the other processor.
Granted, the same program in C/C++ would peg both processors at
100%... but given that the overhead in the interpreter cannot explain
the extra usage, I assume the code in my thread's run functions is
actually executing non-serially.
I assume this is because what I am doing does not require the GIL to
be locked for a significant part of the time my code is running...
what code could I put in my run function to see the behavior I
expected? What code could I put there to take advantage of the
possibility that really the GIL is not locked enough to cause actual
serialization of the threads... anyone care to explain?
More information about the Python-list