[Python-Dev] Removing the GIL (Me, not you!)

["Martin v. Löwis"]

> http://www.artima.com/weblogs/viewpost.jsp?thread=214235) that the
> slowdown was 2x in a single threaded application (which couldn't be due
> to lock contention), it must be due to lock overhead (unless the
> programming was otherwise faulty or there is something else about locks
> that I don't know about - Martin?). Hence I'm assuming that we need to
> reduce lock overhead. If acquiring and releasing locks (part of lock
> overhead) is a simple context switch (and I don't doubt you here), then
> the only remaining thing to optimize is memory operations related to
> lock objects.

I think you are putting too many assumptions on top of each other.
It might also have been that the locks in the slow implementation
were too fine-grained, and that some performance could have been
regained by making them coarser again.

>> You lost me here. What are you trying to achieve? It's not the lock
>> that people complain about, but that Python runs serially most
>> of the time.
> 
> http://en.wikipedia.org/wiki/Lock-free_and_wait-free_algorithms#The_lock-free_approach

The asynchronous model assumes that the sender can continue to process
data without needing a reply. This is not true for the Python threading
model: if the thread needs access to some data structure, it really
needs to wait for the result of that access, because that's the
semantics of the operations.

> Specifically, i'm trying to achieve the approach using a "deposit request".

For that to work, you need to produce a list of requests that can be
processed asynchronously. I can't see any in the Python interpreter.

> I'm also guessing that if
> you do that then for each refcount you're going to have to acquire a
> lock... which happens *very* frequently (and I think by your earlier
> responses you concur).

In that it occurs frequently - not in that you have to acquire a lock
to modify the refcount. You don't.

> Ok. Other people know more about the specifics of the GIL than I do.
> However, the main issue with removing the GIL seems to be the reference
> counting algorithm. 

It isn't. Reference counting could be done easily without the GIL.
It's rather the container objects, and the global variables, that
need protection.

Regards,
Martin