[Python-Dev] GIL removal question

[Vlad Riscutia]

Removing GIL is interesting work and probably multiple people are willing to
contribute. Threading and synchronization is a deep topic and it might be
that if just one person toys around with removing GIL he might not see
performance improvement (not meaning to offend anyone who tried this,
honestly) but what about forking a branch for this work, with some good
benchmarks in place and have community contribute? Let's say first step
would be just replacing GIL with some fine grained locks with expected
performance degradation but afterwards we can try to incrementally improve
on this.

Thank you,
Vlad

On Wed, Aug 10, 2011 at 8:20 AM, Maciej Fijalkowski <fijall at gmail.com>wrote:

> On Wed, Aug 10, 2011 at 1:43 PM, Guido van Rossum <guido at python.org>
> wrote:
> > On Wed, Aug 10, 2011 at 7:32 AM, David Beazley <dave at dabeaz.com> wrote:
> >>
> >> On Aug 10, 2011, at 6:15 AM, Nick Coghlan wrote:
> >>
> >>> On Wed, Aug 10, 2011 at 9:09 PM, David Beazley <dave at dabeaz.com>
> wrote:
> >>>> You're forgetting step 5.
> >>>>
> >>>> 5. Put fine-grain locks around all reference counting operations (or
> rewrite all of Python's memory management and garbage collection from
> scratch).
> >>> ...
> >>>> After implementing the aforementioned step 5, you will find that the
> performance of everything, including the threaded code, will be quite a bit
> worse.  Frankly, this is probably the most significant obstacle to have any
> kind of GIL-less Python with reasonable performance.
> >>>
> >>> PyPy would actually make a significantly better basis for this kind of
> >>> experimentation, since they *don't* use reference counting for their
> >>> memory management.
> >>>
> >>
> >> That's an experiment that would pretty interesting.  I think the real
> question would boil down to what *else* do they have to lock to make
> everything work.   Reference counting is a huge bottleneck for CPython to be
> sure, but it's definitely not the only issue that has to be addressed in
> making a free-threaded Python.
> >
> > They have a specific plan, based on Software Transactional Memory:
> >
> http://morepypy.blogspot.com/2011/06/global-interpreter-lock-or-how-to-kill.html
> >
> > Personally, I'm not holding my breath, because STM in other areas has
> > so far captured many imaginations without bringing practical results
> > (I keep hearing about it as this promising theory that needs more work
> > to implement, sort-of like String Theory in theoretical physics).
>
> Note that the PyPy's plan does *not* assume the end result will be
> comparable in the single-threaded case. The goal is to be able to
> compile two *different* pypy's, one fast single-threaded, one
> gil-less, but with a significant overhead. The trick is to get this
> working in a way that does not increase maintenance burden. It's also
> research, so among other things it might not work.
>
> Cheers,
> fijal
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/riscutiavlad%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110810/6103014a/attachment.html>