[Cython] 'with gil:' statement

mark florisson markflorisson88 at gmail.com
Wed Mar 16 13:55:44 CET 2011

On 16 March 2011 13:37, Dag Sverre Seljebotn <d.s.seljebotn at astro.uio.no> wrote:
> On 03/16/2011 12:54 PM, mark florisson wrote:
>> On 16 March 2011 11:58, Dag Sverre Seljebotn<d.s.seljebotn at astro.uio.no>
>>  wrote:
>>> On 03/16/2011 11:28 AM, mark florisson wrote:
>>> I implemented the 'with gil:' statement, and have added error checking
>>> for nested 'with gil' or 'with nogil' statements. For instance, with
>>> the patch applied Cython wil issue an error when you have e.g.
>>> with nogil:
>>>     with nogil:
>>>         ...
>>> (or the same thing for 'with gil:'). This because nested 'nogil' or
>>> 'gil' blocks, without intermediate GIL-acquiring/releasing, will abort
>>> the Python interpreter. However, if people are acquiring the GIL
>>> manually in nogil blocks and then want to release the GIL with a
>>> 'nogil' block, it will incorrectly issue an error. I do think this
>>> kind of code is uncommon, but perhaps we want to issue a warning
>>> instead?
>>> I think we should make nested nogil-s noops, i.e.
>>> with nogil:
>>>     with nogil: # =>  if True:
>>> This is because one may want to change "with nogil" to "with gil" for
>>> debugging purposes (allow printing debug information).
>> Interesting, that does sound convenient, but I'm not if mere
>> convenience should move us to simply ignore what is technically most
>> likely incorrect code (unless there is intermediate manual locking).
> I'm not sure if I understand what you mean here. How does simply ignoring
> redundant "with [no]gil" statements cause incorrect code? Or do you mean
> this is a
> I'm just trying to minimize the "language getting in your way" factor. It is
> pretty useless to write
> if x:
>    if x:
>        ...
> as well, but Python does allow it.
> Warnings on nested "with nogil" is more the role of a "cylint" in my
> opinion.

Perhaps you're right. However, I just think it is important for users
to realize that in general, they cannot unblock threads recursively.
Currently the error checking code catches multiple nested 'with
(no)gil', but it doesn't catch this:

cdef void func() nogil:
    with nogil:

with nogil:

But the problem is that it does abort the interpreter. So I thought
that perhaps emphasizing that that code is incorrect for at least the
easy-to-catch cases, we might make users somewhat more aware. Because
if the above code aborts Python, but a nested 'with nogil:' is valid
code, there might be a source for confusion.

>> In any case, I wouldn't really be against that. If you simply want to
>> allow this for debugging, we could also allow print statements in
>> nogil sections, by either rewriting it using 'with gil:', or by
>> inserting a simple printf (in which case you probably want to place a
>> few restrictions).
> It's not only print statements. I.e., if I think something is wrong with an
> array, I'll stick in code like
> print np.std(x), np.mean(x), np.any(np.isnan(x))
> or something more complex that may require temporaries. Or even plot the
> vector:
> plt.plot(x)
> plt.show() # blocks until I close plot window
> Or, launch a debugger:
> if np.any(np.isnan(x)):
>     import pdb; pdb.set_trace()
> so I guess I should stop saying this is only about printing. In general,
> it's nice to use Python during debugging. I find myself replacing "with
> nogil" with "if True" in such situations, to avoid reindenting.
> I guess I can soon start using "with gil" around the debug code though.
> Again, the restriction on nested nogil/gil is not a big problem, just an
> instance of "the language getting in your way".

I understand. If your code doesn't introduce Cython-level blocks, you
could at least put it on one line so you can easily comment it out.

> Dag
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel

More information about the cython-devel mailing list