[Python-Dev] Removing the GIL (Me, not you!)

[Adam Olsen]

On 9/13/07, Hrvoje Nikšić <hrvoje.niksic at avl.com> wrote:
> On Thu, 2007-09-13 at 13:15 +0200, "Martin v. Löwis" wrote:
> > > To put it another way, would it actually matter if the reference
> > > counts for such objects became hopelessly wrong due to non-atomic
> > > adjustments?
> >
> > If they drop to zero (which may happen due to non-atomic adjustments),
> > Python will try to release the static memory, which will crash the
> > malloc implementation.
>
> More precisely, Python will call the deallocator appropriate for the
> object type.  If that deallocator does nothing, the object continues to
> live.  Such objects could also start out with a refcount of sys.maxint
> or so to ensure that calls to the no-op deallocator are unlikely.
>
> The part I don't understand is how Python would know which objects are
> global/static.  Testing for such a thing sounds like something that
> would be slower than atomic incref/decref.

I've explained my experiments here:
http://www.artima.com/forums/flat.jsp?forum=106&thread=214235&start=30&msRange=15#279978

Basically though, atomic incref/decref won't work.  Once you've got
two threads modifying the same location the costs skyrocket.  Even
without being properly atomic you'll get the same slowdown on x86
(who's cache coherency is fairly strict.)

The only two options are:
A) Don't modify an object on every incref/decref.  Deletion must be
delayed.  This lets you share (thread-safe) objects.
B) Don't share *any* objects.  This is a process model (even if
they're lightweight like erlang).  For the near future, it's much
easier to do this using real processes though.

Threading is much more powerful, but it remains to be proven that it
can be done efficiently.

-- 
Adam Olsen, aka Rhamphoryncus