[Python-Dev] Removing the GIL (Me, not you!)

[Tony Nelson]

At 1:51 AM -0500 9/14/07, Justin Tulloss wrote:
>On 9/14/07, Adam Olsen <<mailto:rhamph at gmail.com>rhamph at gmail.com> wrote:
>
>> Could be worth a try. A first step might be to just implement
>> the atomic refcounting, and run that single-threaded to see
>> if it has terribly bad effects on performance.
>
>I've done this experiment.  It was about 12% on my box.  Later, once I
>had everything else setup so I could run two threads simultaneously, I
>found much worse costs.  All those literals become shared objects that
>create contention.
>
>
>It's hard to argue with cold hard facts when all we have is raw
>speculation. What do you think of a model where there is a global "thread
>count" that keeps track of how many threads reference an object? Then
>there are thread-specific reference counters for each object. When a
>thread's refcount goes to 0, it decrefs the object's thread count. If you
>did this right, hopefully there would only be cache updates when you
>update the thread count, which will only be when a thread first references
>an object and when it last references an object.

It's likely that cache line contention is the issue, so don't glom all the
different threads' refcount for an object into one vector.  Keep each
thread's refcounts in a per-thread vector of objects, so only that thread
will cache that vector, or make refcounts so large that each will be in its
own cache line (usu. 64 bytes, not too horrible for testing purposes).  I
don't know all what would be required for separate vectors of refcounts,
but each object could contain its index into the vectors, which would all
be the same size (Go Virtual Memory!).


>I mentioned this idea earlier and it's growing on me. Since you've
>actually messed around with the code, do you think this would alleviate
>some of the contention issues?
>
>Justin

Your idea can be combined with the maxint/2 initial refcount for
non-disposable objects, which should about eliminate thread-count updates
for them.
-- 
____________________________________________________________________
TonyN.:'                       <mailto:tonynelson at georgeanelson.com>
      '                              <http://www.georgeanelson.com/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20070914/70897382/attachment.htm