GIL in alternative implementations

[Dino Viehland]

In IronPython we have fine grained locking on our mutable data structures.  In particular we have a custom dictionary type which is designed to allow lock-free readers on common operations while writers take a lock.  Our list implementation is similar but in some ways that's trickier to pull off due to features like slicing so if I recall correctly we only have lock-free reads when accessing a single element.

For .NET data structures they follow the .NET convention which is up to the data structure.  So if you wanted to get every last bit of performance out of your app you could handle thread safety yourself and switch to using the .NET dictionary or list types (although they're a lot less friendly to Python developers).

Because of these locks on micro-benchmarks that involve simple list/dict manipulations you do see noticeably worse performance in IronPython vs. CPython.  http://ironpython.codeplex.com/wikipage?title=IP27A1VsCPy27Perf&referringTitle=IronPython%20Performance  - See the SimpleListManipulation and SimpleDictManipulation as the core examples here.  Also CPython's dictionary is so heavily tuned it's hard to beat anyway, but this is a big factor.

Finally one of the big differences with both Jython and IronPython is that we have good garbage collectors which don't rely upon reference counting.  So one area where CPython gains from having a GIL is a non-issue for us as we don't need to protect ref counts or use interlocked operations for ref counting.

From: python-list-bounces+dinov=exchange.microsoft.com at python.org [mailto:python-list-bounces+dinov=exchange.microsoft.com at python.org] On Behalf Of Pascal Chambon
Sent: Friday, May 27, 2011 2:22 PM
To: python-list at python.org >> Python List
Subject: GIL in alternative implementations

Hello everyone,

I've already read quite a bit about the reasons for the GIL in CPython, i.e to summarize, that a more-fine graine locking, allowing real concurrency in multithreaded applications, would bring too much overhead for single-threaded python applications.

However, I've also heard that other python implementations (ironpython, jython...) have no GIL, and yet nobody blames them for performance penalties that would be caused by that lack (I especially think about IronPython, whose performances compare quite well to CPython).

So I'd like to know: how do these other implementations handle concurrency matters for their primitive types, and prevent them from getting corrupted in multithreaded programs (if they do) ? I'm not only thinking about python types, but also primitive containers and types used in .Net and Java VMs, which aren't atomic elements either at an assembly-level point of view.

Do these VMs have some GIL-like limitations, that aren't spoken about ? Are there functionings completely different from the CPython VM, so that the question is not relevant ? Do people consider that they always concern multithreaded applications, and so accept performance penalties that they wouldn't allow in their CPython scripts ?

I think you in advance for your lights on these questions.

Regards,
Pkl

[[ Important Note: this is a serious question, trolls and emotionally disturbed persons had better go on their way. ]]


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20110527/6a27afc4/attachment.html>