[Python-Dev] Thoughts fresh after EuroPython

[Ronald Oussoren]

On 26 Jul, 2010,at 12:00 PM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:

On 26/07/2010 04:42, Guido van Rossum wrote:
> On Sun, Jul 25, 2010 at 8:31 PM, Peter Portante
> <peter.a.portante at gmail.com> wrote:
> 
>> FWIW: We use Python at Tabblo, straddled across Python 2.5.4 and 2.6.5. They
>> work. And they work well. But we make light use of threads (mostly
>> background I/O handling), and heavy use of multiple processes because we
>> can't take advantage of our multi-core systems otherwise.
>> 
> Isn't this an indication that the GIL is, in fact, not (much of) a problem?
>
> I wish those trying to get rid of the GIL well But it may not be the
> panacea some folks are hoping for. Multi-threaded programming remains
> hard (and removing the GIL might actually make it harder).
>
> Jython and IronPython don't have a GIL, and I think PyPy may not
> either. Does anyone have experience with GIL-free programming in one
> of those?
>
> 
At Resolver Systems we created a "calculation system" that does large 
calculations on background threads using IronPython. Doing them on a 
background thread allows the ui to remain responsive. Several 
calculations could run simultaneously using multiple cores.

As the calculation operates on a large object graph (which the ui then 
needs access to in order to display it) using multiprocessing would have 
imposed a very big overhead due to serialization / deserialization (the 
program runs on windows).

Using CPython would have made the program a lot slower due to the GIL.
 

I have a simular usecase, although in my case it's more using large blobs instead of complicated datastructures. I'm not hurt by the GIL because most threads run C code most of the time, which enables us to use multiple CPU cores without getting hurt by the GIL.

In my opinion the GIL is a weak point of CPython and it would be nice if it could be fixed.  That is however easier said than done, a number of people have tried in the past and ran into implementation limitations like our refcounting garbage collector that make hard to remove the GIL without either rewriting lots of code, or running into a brick wall performance-wise.

The HotPy presentation at EuroPython shows that it is possible to remove the GIL, although at the cost of replacing the garbage collector and most likely breaking existing C extensions (although the HotPy author seemed to have a possible workaround for that)

Ronald
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100726/52cf2592/attachment.html>