[Cython] 'with gil:' statement

[Dag Sverre Seljebotn]

On 03/17/2011 11:16 AM, mark florisson wrote:
> On 17 March 2011 10:08, Dag Sverre Seljebotn<d.s.seljebotn at astro.uio.no>  wrote:
>>
>> How about this compromise: We balk on the code you wrote with:
>>
>> Error line 345: Exceptions propagating from "with gil" block cannot be
>> propagated out of function, please insert try/except and handle exception
>>
>> So that we require this:
>>
>> with gil:
>>     try:
>>         ...
>>     except:
>>         warnings.warning(...) # or even cython.unraisable(e)
>>
>> This keeps me happy about not abusing the with statement for strange control
>> flow, and makes the "with gil" useful for raising exceptions inside regular
>> def functions with nogil blocks.
>>
> I agree with your previous statement, but not with your compromise :).
> We have to differentiate between two cases, similar to Stefan's cases,
> but different in a very important way that matter for nested GIL
> blocks.
>
> 1) Exceptions can propagate to some outer GIL section (in or outside
> the current function)
> 2) Exceptions can't propagate, because there is no outer GIL section
> and the function has a non-object return type
>
> With your compromise, with 1) exceptions cannot propagate, but with 2)
> you win forcing the user to be explicit. But then you still need to
> write to some variable indicating that an exception occurred and
> adjust control flow accordingly in your nogil section (unless you want
> to clean up and return immediately).
>
> If you have Python with-statement semantics, you can do the following,
> for instance:
>
> cdef void func() nogil:
>      with gil:
>          try:
>
>              with nogil:
>                  with gil:
>                      code that may raise an exception
>
>                  this is not executed
>
>          except ExceptionRaisedFromInnerWithGilBlock:
>              handle exception here
>
> The point is, if you have case 2), and you want to use GIL code, you
> need to handle exceptions in some way. Forcing the user to not
> propagate anything doesn't sound right, unless this holds only for the
> outermost 'with gil' block. I would be OK with that, although it would
> be inconsistent with how exceptions in normal cdef functions with
> non-object return work, so I would say that we'd have to force it in
> the same manner there.

I think we should perhaps look at enforcing explicit exception-ignoring 
everywhere.. there's a lot of details to hash out, and there's the issue 
of backwards compatability, but it could be dealt with with a couple of 
releases where we only raise a warning and so on.

It could involve a *very* limited subset of exception handling for use 
in nogil mode (i.e., only a bare "except:" statement allowed, where one 
can call either "cython.unraisable()", "pass", or set a flag).

Dag Sverre