[Python-Dev] Timeout for PEP 550 / Execution Context discussion

[Nick Coghlan]

On 18 October 2017 at 16:25, Ethan Furman <ethan at stoneleaf.us> wrote:

> On 10/17/2017 09:40 PM, Nick Coghlan wrote:
>
>> At the Python API layer, we don't expose the ability to switch explicitly
>> to another thread state while remaining within
>> the current function. Instead, we only offer two options: starting a new
>> thread, and waiting for a thread to finish
>> execution. The lifecycle of the thread local storage is then
>> intrinsically linked to the lifecycle of the thread it
>> belongs to.
>>
>
> I seem to remember mention about frameworks being able to modify contexts
> for various tasks/coroutines; if the framework cannot create and switch to
> a new context how will it set them up?  Or did I misunderstand?


That's what Context.run() will handle.

>From a usage perspective, what Yury and I are suggesting is that switching
execution to a new context should always implicitly switch back when the
called operation returns:

    def run_in_isolated_ec(operation):
        ec = contextvars.new_context()
        return ec.run(operation)

That's safe and simple to use, and integrates nicely with the APIs for
resuming coroutines (next(cr), cr.__next__(), cr.send(), cr.throw()).

By contrast, exposing set_ctx() directly puts the responsibility for
reverting the active context back to the initial one on the caller:

    def run_in_isolated_ec(operation):
        ec = contextvars.new_context()
        initial_ec = contextvar.get_ctx()
        contextvars.set_ctx(ec)
        try:
            return operation()
        finally:
            # What happens if we forget to revert back to the previous
context here?
            contextvars.set_ctx(initial_ec)

While the contextvars implementation is going to need a context switching
capability like that internally in order to implement ec.run() (just as the
threading implementation's C API includes PyThreadState_Swap), we don't
currently have a use case for exposing it as a Python level API.

And if we don't expose an API that allows it, then we can delay specifying
what effect the following code should have all the calling function or
coroutine (or whether it's a runtime error):

    def buggy_run_in_isolated_ec(operation):
        ec = contextvars.new_context()
        contextvars.set_ctx(ec)
        return operation()
        # Oops, forgot to revert the active context to the one we were
called with

So if we start out only exposing "Context.run()" at the Python layer
(covering all currently known use cases), then any subsequent addition of
an in-place context switching API can be guided by specific examples of
situations where Context.run() isn't sufficient.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20171018/686694b7/attachment.html>