[Cython] [cython-users] Calling gil-requiring function not allowed without gil

[Stefan Behnel]

[moving this away from cython-users]

Dag Sverre Seljebotn, 11.08.2011 14:53:
> On 08/11/2011 02:13 PM, Stefan Behnel wrote:
>> Dag Sverre Seljebotn, 11.08.2011 13:58:
>>> On 08/11/2011 01:43 PM, Ting Zhou wrote:
>>>> here is my pyx file. When I compile it, it says "Calling gil-requiring
>>>> function not allowed without gil" at calling dabs(x[i]). Why my dabs
>>>> function is a gil-requiring function?
>>>>
>>>> ====================================
>>>> from cython.parallel import *
>>>> import numpy as np
>>>> cimport numpy as np
>>>> cimport cython
>>>>
>>>> cdef double dabs(double x):
>>>
>>> This should be
>>>
>>> cdef double dabs(double x) nogil:
>>
>> Note that Cython cannot infer this automatically. Even a trivial
>> function may require an exclusive global lock for some reason, and it's
>> common to use the GIL for that. So the programmer must be explicit here.
>
> Are you still against this mini-CEP?:
>
> with cython.global_lock():
> ...
>
> Where global_lock() is GIL-requiring noop.
>
> This
>
> a) Improves code readability vastly. Having a critical section take effect
> because of the *lack* of a keyword is just very odd to anyone who's not
> shoulder deep in CPython internals

The GIL is more than a critical section. It's just there - and that's a 
good thing. Just take it as a part of the runtime environment, and disable 
it when you are sure you can *safely* do without it. A global lock makes 
life a lot easier, as it prevents unconditional (and unexpected) concurrency.


> b) Allows inferring 'nogil'

No it doesn't, because existing code doesn't use it. Only because you use 
one little statement in one part of your sources doesn't mean you know what 
you're doing, and that Cython can happily assume reversed semantics for the 
rest. Don't forget that threading is a dangerous abstraction, hard to get 
right and likely even the most error prone concurrency mechanism in 
widespread use. Not everything is "trivially parallelisable", and even 
those cases that are still produce enough problems.

Actually, it's worth asking if even the prange loop was a good idea, given 
the amount of confusion driven traffic that it currently produces on the 
mailing list. I'm not arguing against it, but it seems to be a trickier 
concept than it appears at first sight. IMHO, that's worth working on 
*before* we introduce even more hard-to-understand concepts and 
hard-to-control compiler trickery.

Stefan