[Python-Dev] Fwd: Removal of GIL through refcounting removal.

Adam Olsen rhamph at gmail.com
Fri Oct 31 09:29:24 CET 2008

On Fri, Oct 31, 2008 at 2:11 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Adam Olsen wrote:
>> On Fri, Oct 31, 2008 at 12:24 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>> Neil Schemenauer wrote:
>>>> Sigurd Torkel Meldgaard <stm at daimi.au.dk> wrote:
>>>>> For a student project in a course on virtual machines, we are
>>>>> evaluating the possibility to experiment with removing the GIL
>>>>> from CPython
>>>> Hi,
>>>> It's great to hear of this kind of project.  I think what you want
>>>> to do is difficult but possible.  The major compilcation would be
>>>> that extension modules would have to re-written since they all
>>>> assume a reference counting GC.  A foreign function interface like
>>>> CMU Lisp's "alien" or GHC's FFI is not necessarily any worse but it
>>>> does place different demands on extension module authors.
>>> Michael Foord's comment about the way Ironclad does it suggests a
>>> possible interim step that would allow existing extensions to be used at
>>> the cost of some additional overhead: use free threading in the core,
>>> but have an "extension module lock" that cuts things back to a single
>>> thread whenever non-core code is invoked.
>> Ahh, or do it per object!  Convert the core to use a precise GC
>> references, but retain the refcount API as a sort of forced unknown
>> reference.  Needs bodging to retain our current handling of cycles in
>> refcounted objects, but it should be doable, and it potentially has a
>> lot less overhead (and is more scalable) than what I've got now with
>> safethread.
> The GIL protects more than just refcounting: it is used widely to
> protect access to shared data structures both inside and outside the
> core. Until all the extension modules implement their own locks instead
> of relying on the GIL, some form of global lock needs to remain if you
> want to preserve compatibility with existing extensions. And since being
> able to use existing extensions is the main advantage of using CPython
> rather than just writing a separate free-threaded Python VM, that seems
> like a fairly important design goal.

safethread already handles that, via monitors.  The main thread is
given a monitor that all old, unshareable extension modules exist in.
Once they're updated and marked as shareable they can be used from any
monitor in any thread.

Adam Olsen, aka Rhamphoryncus

More information about the Python-Dev mailing list