[Cython] 'with gil:' statement

[mark florisson]

On 16 March 2011 11:58, Dag Sverre Seljebotn <d.s.seljebotn at astro.uio.no> wrote:
> On 03/16/2011 11:28 AM, mark florisson wrote:
>
> I implemented the 'with gil:' statement, and have added error checking
> for nested 'with gil' or 'with nogil' statements. For instance, with
> the patch applied Cython wil issue an error when you have e.g.
>
> with nogil:
>     with nogil:
>         ...
>
> (or the same thing for 'with gil:'). This because nested 'nogil' or
> 'gil' blocks, without intermediate GIL-acquiring/releasing, will abort
> the Python interpreter. However, if people are acquiring the GIL
> manually in nogil blocks and then want to release the GIL with a
> 'nogil' block, it will incorrectly issue an error. I do think this
> kind of code is uncommon, but perhaps we want to issue a warning
> instead?
>
> I think we should make nested nogil-s noops, i.e.
>
> with nogil:
>     with nogil: # => if True:
>
> This is because one may want to change "with nogil" to "with gil" for
> debugging purposes (allow printing debug information).

Interesting, that does sound convenient, but I'm not if mere
convenience should move us to simply ignore what is technically most
likely incorrect code (unless there is intermediate manual locking).
In any case, I wouldn't really be against that. If you simply want to
allow this for debugging, we could also allow print statements in
nogil sections, by either rewriting it using 'with gil:', or by
inserting a simple printf (in which case you probably want to place a
few restrictions).

> Another feedback is that I wonder whether we should put the "gil" and
> "nogil" psuedo-context managers both in cython namespace, and sort of
> deprecate the "global" nogil, rather than introduce yet another name that
> can't be used safely for all kinds of variables.

Hmm, good catch. Actually, 'with cython.nogil:' is already possible,
but cython.gil isn't (in fact, 'with cython.somethingrandom:' seems to
simply be ignored).
So I guess I'll have to fix that :)

> -- Dag
>
> The 'with gil:' statement can now be used in the same way as 'with
> nogil:'. Exceptions raised from GIL blocks will be propagated if
> possible (i.e., if the return type is 'object'). Otherwise it will
> jump to the end of the function and use the usual
> __Pyx_WriteUnraisable, there's not really anything new there.
>
> For functions declared 'nogil' that contain 'with gil:' statements, it
> will safely lock around around the initialization of any Python
> objects and set up the refnanny context (with appropriate preprocessor
> guards). At the end of the function it will safely lock around the
> teardown of the refnanny context. With 'safely' I mean that it will
> create a thread state if it was not already created and may be called
> even while the GIL is already held (using PyGILState_Ensure()). This
> means variables are declared and initialized in the same way as in
> normal GIL-holding functions (except that there is additional
> locking), and of course the GIL-checking code ensures that errors are
> issued if those variables are attempted to be used outside any GIL
> blocks.
>
> Could someone review the patch (which is attached)? Maybe check if I
> haven't missed any side cases and such?
>
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel
>
>
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel
>
>