[Python-Dev] Fwd: Removal of GIL through refcounting removal.

[Nick Coghlan]

Adam Olsen wrote:
> On Fri, Oct 31, 2008 at 12:24 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Neil Schemenauer wrote:
>>> Sigurd Torkel Meldgaard <stm at daimi.au.dk> wrote:
>>>> For a student project in a course on virtual machines, we are
>>>> evaluating the possibility to experiment with removing the GIL
>>>> from CPython
>>> Hi,
>>>
>>> It's great to hear of this kind of project.  I think what you want
>>> to do is difficult but possible.  The major compilcation would be
>>> that extension modules would have to re-written since they all
>>> assume a reference counting GC.  A foreign function interface like
>>> CMU Lisp's "alien" or GHC's FFI is not necessarily any worse but it
>>> does place different demands on extension module authors.
>> Michael Foord's comment about the way Ironclad does it suggests a
>> possible interim step that would allow existing extensions to be used at
>> the cost of some additional overhead: use free threading in the core,
>> but have an "extension module lock" that cuts things back to a single
>> thread whenever non-core code is invoked.
> 
> Ahh, or do it per object!  Convert the core to use a precise GC
> references, but retain the refcount API as a sort of forced unknown
> reference.  Needs bodging to retain our current handling of cycles in
> refcounted objects, but it should be doable, and it potentially has a
> lot less overhead (and is more scalable) than what I've got now with
> safethread.

The GIL protects more than just refcounting: it is used widely to
protect access to shared data structures both inside and outside the
core. Until all the extension modules implement their own locks instead
of relying on the GIL, some form of global lock needs to remain if you
want to preserve compatibility with existing extensions. And since being
able to use existing extensions is the main advantage of using CPython
rather than just writing a separate free-threaded Python VM, that seems
like a fairly important design goal.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------