[Cython] Acquisition counted cdef classes

[Stefan Behnel]

Greg Ewing, 26.10.2011 00:27:
> Dag Sverre Seljebotn wrote:
>
>> I'd gladly take a factor two (or even four) slowdown of CPython code any
>> day to get rid of the GIL :-). The thing is, sometimes one has 48 cores
>> and consider a 10x speedup better than nothing...
>
> Another thing to consider is that locking around refcount
> changes may not be as expensive in typical Cython code as
> it is in Python.
>
> The trouble with Python is that you can't so much as scratch
> your nose without touching a big pile of ref counts. But
> if the Cython code is only dealing with a few Python objects
> and doing most of its work at the C level, the relative
> overhead of locking around refcount changes may not be
> significant.
>
> So it may be worth trying the strategy of just acquiring
> the GIL whenever a refcount needs to be changed in a nogil
> section, and damn the consequences.

Hmm, interesting. That would give new semantics to "nogil" sections, basically:

"""
You can do Python interaction in nogil code, however, this will slow down 
your code. Cython will generate C code to acquire and release the GIL 
around any Python interaction that your code performs, thus serialising any 
calls into the CPython runtime. If you want to avoid this serialisation, 
use "cython -a" to find out where Python interaction happens and use static 
typing to let Cython generate C code instead.
"""

In other words: "with gil" sections hold the GIL by default and give it 
away on explicit request, whereas "nogil" sections have the GIL released by 
default and acquire it on implicit need.

The advantage over object level locking is that this does not increase the 
in-memory size of the object structs, and that it works with *any* Python 
object, not just extension types with a compile time known type.

I kind of like that.

Stefan