[IronPython] IronPython cs Cpython wrt GIL

Nicolas Lehuen nicolas at lehuen.com
Thu Oct 14 11:54:14 CEST 2004


> Nicolas Lehuen wrote:
> > There has been yet another thread on the Global Interpreter 
> Lock found 
> > in Cpython on comp.lang.python. Is there any such "annoyance" in 
> > IronPython (and, come to think of it, Jython) ? If not, I guess it 
> > would be a big advantage of IronPython (or Jython) vs CPython on 
> > multi-CPU or hyperthreading CPUs ?
> 
> IronPython has no GIL today and it's unlikely that it will 
> ever have one.
> This could be useful for running multi-threaded Python 
> programs.  The IronPython core itself should be mostly 
> thread-safe; however, that hasn't been tested so there are 
> almost certainly some issues there.  I have full confidence 
> that the IronPython core functionality will have no 
> difficulty becoming fully thread safe as it moves to 1.0.
> 
> My area of concern is around the standard objects.  
> IronPython lists and dictionaries are not currently 
> thread-safe.  This is in line with the CLR collection APIs 
> that assume the developer using lists and dictionaries will 
> handle any needed synchronization.  However, because of 
> Python's GIL, Python lists and dictionaries behave as if they 
> are thread-safe to the Python programmer.  For compatibility 
> with multi-threaded Python programs, IronPython will probably 
> need to use thread-safe lists and dicts for these standard 
> Python objects and this will impose some small but noticeable 
> performance overhead.  I'd like to come up with a more clever 
> solution to this dilemma so that synchronization overhead is 
> only imposed when it's truly needed, but I can't come up with 
> a solution that wouldn't inflict undue compatibility burdens 
> on Python devs.
> 
> -Jim

Hi Jim,

I'm not sure that lists and dictionaries behave as if they are thread-safe.
What's sure is that a lot of Python developer act as if they are ;). I've
joined a recent comp.lang.python thread where I ranted about the
quasi-prehistoric thread support in Python, quite representative of the fact
that a lot of Python developer don't worry about locking issues (as an
illustration, a very core tool such as a reentrant lock is still implemented
in pure Python, see threading.Rlock ; this should definitely be implemented
in C).

I think that the Python locking model for lists is quite the same as for
Java's Vector (not ArrayList or a synchronized ArrayList), that is to say
that you can safely assume that multiple threads can concurrently read or
write in the structure without breaking it (which could append if two
threads tried to reallocate the underlying array to a greater size, for
example, but the GIL makes sure that won't never append).

But this assumption won't be sufficient to make sure that multiple threads
using the same list can have a consistent behaviour with respect to the
algorithms they implement ; developers need to explicitely use locking
semantics, be it in Python, Java or C#. Enter the dreadful problem of having
to write code like :

object.lock.acquire()
try:
	# do something
finally:
	object.lock.release()

Whereas in Java you'd just write :

synchronized(object) {
	// do something
}

The GIL is purely a design choice : having a big lock vs a lock per object
and synchronisation to make sure that objects are kept in a coherent state.
This seems a big difference, but at the end, for the developper, the
behaviour of the language is nearly the same, since all native code is
executed atomically from a byte-code point of view.

The good news is that without the GIL, we can expect IronPython to perform
even better than Cpython on multi-CPU / multi-core CPU machines.

Regards,

Nicolas




More information about the Ironpython-users mailing list