[Python-Dev] GIL removal question

[Guido van Rossum]

On Wed, Aug 10, 2011 at 7:32 AM, David Beazley <dave at dabeaz.com> wrote:
>
> On Aug 10, 2011, at 6:15 AM, Nick Coghlan wrote:
>
>> On Wed, Aug 10, 2011 at 9:09 PM, David Beazley <dave at dabeaz.com> wrote:
>>> You're forgetting step 5.
>>>
>>> 5. Put fine-grain locks around all reference counting operations (or rewrite all of Python's memory management and garbage collection from scratch).
>> ...
>>> After implementing the aforementioned step 5, you will find that the performance of everything, including the threaded code, will be quite a bit worse.  Frankly, this is probably the most significant obstacle to have any kind of GIL-less Python with reasonable performance.
>>
>> PyPy would actually make a significantly better basis for this kind of
>> experimentation, since they *don't* use reference counting for their
>> memory management.
>>
>
> That's an experiment that would pretty interesting.  I think the real question would boil down to what *else* do they have to lock to make everything work.   Reference counting is a huge bottleneck for CPython to be sure, but it's definitely not the only issue that has to be addressed in making a free-threaded Python.

They have a specific plan, based on Software Transactional Memory:
http://morepypy.blogspot.com/2011/06/global-interpreter-lock-or-how-to-kill.html

Personally, I'm not holding my breath, because STM in other areas has
so far captured many imaginations without bringing practical results
(I keep hearing about it as this promising theory that needs more work
to implement, sort-of like String Theory in theoretical physics).

But I'm also not denying that Armin Rigo has a brain the size of the
planet, and PyPy *has* already made much real, practical progress.

-- 
--Guido van Rossum (python.org/~guido)