[Python-ideas] New PEP 550: Execution Context

[Nick Coghlan]

On 12 August 2017 at 08:37, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
> Hi,
>
> This is a new PEP to implement Execution Contexts in Python.
>
> The PEP is in-flight to python.org, and in the meanwhile can
> be read on GitHub:
>
> https://github.com/python/peps/blob/master/pep-0550.rst
>
> (it contains a few diagrams and charts, so please read it there.)

The fully rendered version is also up now:
https://www.python.org/dev/peps/pep-0550/

Thanks for this! The general approach looks good to me, so I just have
some questions about specifics of the API:

1. Are you sure you want to expose the CoW type to pure Python code?

The draft API looks fairly error prone to me, as I'm not sure of the
intended differences in behaviour between the following:

    @contextmanager
    def context(x):
        old_x = sys.get_execution_context_item('x')
        sys.set_execution_context_item('x', x)
        try:
            yield
        finally:
            sys.set_execution_context_item('x', old_x)

    @contextmanager
    def context(x):
        old_x = sys.get_execution_context().get('x')
        sys.get_execution_context()['x'] = x
        try:
            yield
        finally:
            sys.get_execution_context()['x'] = old_x

    @contextmanager
    def context(x):
        ec = sys.get_execution_context()
        old_x = ec.get('x')
        ec['x'] = x
        try:
            yield
        finally:
            ec['x'] = old_x

It seems to me that everything would be a lot safer if the *only*
Python level API was a live dynamic view that completely hid the
copy-on-write behaviour behind an "ExecutionContextProxy" type, such
that the last two examples were functionally equivalent to each other
and to the current PEP's get/set functions (rendering the latter
redundant, and allowing it to be dropped from the PEP).

If Python code wanted a snapshot of the current state, it would need
to call sys.get_execution_context().copy(), which would give it a
plain dictionary containing a shallow copy of the execution context at
that particular point in time.

If there's a genuine need to expose the raw copy-on-write machinery to
Python level code (e.g. for asyncio's benefit), then that could be
more clearly marked as "here be dragons" territory that most folks
aren't going to want to touch (e.g. "sys.get_raw_execution_context()")

2. Do we need an ag_isolated_execution_context for asynchronous
generators? (Modify this question as needed for the answer to the next
question)

3. It bothers me that *_execution_context points to an actual
execution context, while *_isolated_execution_context is a boolean.
With names that similar I'd expect them to point to the same kind of
object.

Would it work to adjust that setting to say that rather than being an
"isolated/not isolated" boolean, we instead made it a cr_back reverse
pointer to the awaiting coroutine (akin to f_back in the frame stack),
such that we had a doubly-linked list that defined the coroutine call
stacks via their cr_await and cr_back attributes?

If we did that, we'd have:

  Top-level Task: cr_back -> NULL (C) or None (Python)
  Awaited coroutine: cr_back -> coroutine that awaited this one (which
would in turn have a cr_await reference back to here)

coroutine.send()/throw() would then save and restore the execution
context around the call if cr_back was NULL/None (equivalent to
isolated==True in the current PEP), and leave it alone otherwise
(equivalent to isolated==False).

For generators, gi_back would normally be NULL/None (since we don't
typically couple regular generators to a single managing object), but
could be set appropriately by types.coroutine when the generator-based
coroutine is awaited, and by contextlib.contextmanager before starting
the underlying generator. (It may even make sense to break the naming
symmetry for that attribute, and call it something like "gi_owner",
since generators don't form a clean await-based logical call chain the
way native coroutines do).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia