[Python-ideas] PEP 550 v2

[Nick Coghlan]

On 17 August 2017 at 02:36, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
> Yeah, this is tricky. The main issue is indeed the confusion of what
> methods you need to call -- "get/set" or
> "get_local_state/set_local_state".
>
> On some level the problem is very similar to regular Python scoping rules:
>
> 1. we have local hames
> 2. we have global names
> 3. we nave 'nonlocal' modifier
>
> IOW scoping isn't easy, and you need to be conscious of what you do.
> It's just that we are so used to these scoping rules that they have a
> low cognitive effort for us.
>
> One of the ideas that I have in mind is to add another level of
> indirection to separate "global get" from "local set/get":
>
> 1. Rename ContextItem to ContextKey (reasoning for that in parallel thread)
>
> 2. Remove ContextKey.set() method
>
> 3. Add a new ContextKey.value() -> ContextValue
>
>     ck = ContextKey()
>
>     with ck.value() as val:
>         val.set(spam)
>         yield
>
> or
>
>      val = ck.value()
>      val.set(spam)
>      try:
>           yield
>      finally:
>           val.clear()
>
> Essentially ContextValue will be the only API to set values in
> execution context. ContextKey.get() will be used to get them.
>
> Nathaniel, Nick, what do you guys think?

I think I don't want to have try to explain to anyone what happens if
I get a context value in my current execution environment and then
send that value reference into a different execution context :)

So I'd prefer my earlier proposal of:

    # Resolve key in current execution environment
    ck.get_value()
    # Assign to key in current execution context
    ck.set_value(value)
    # Assign to key in specific execution context
    sys.run_with_active_context(ec, ck.set_value, value)

One suggestion I do like is Stefan's one of using "ExecutionContext"
to refer to the namespace that ck.set_value() writes to, and then
"ExecutionEnvironment" for the whole chain that ck.get_value() reads.

Similar to "generator" and "package", we'd still end up with "context"
being inherently ambiguous when used without qualification:

- PEP 550 execution context
- exception handling context (for chained exceptions)
- with statement context
- various context objects, like the decimal context

But we wouldn't have two different kinds of context within PEP 550
itself. Instead, we'd have to start disambiguating the word
environment:

- PEP 550 execution environment
- process environment (i.e. os.environ)

The analogy between process environments and execution environments
wouldn't be exact (since the key-value pairs in process environments
are copied eagerly rather than via lazily chained lookups), but once
you account for that, the parallels between an operating system level
process environment tree and a Python level execution environment tree
as proposed in PEP 550 seem like they would be helpful rather than
confusing.

> [..]
>>> * ``sys.get_execution_context()`` function.  The function returns a
>>>   copy of the current EC: an ``ExecutionContext`` instance.
>>
>> If there are enough of these functions then it might make sense to
>> stick them in their own module instead of adding more stuff to sys. I
>> guess worrying about that can wait until the API details are more firm
>> though.
>
> I'm OK with this idea -- pystate.c becomes way too crowded.
>
> Maybe we should just put this stuff in _contextlib.c and expose in the
> contextlib module.

Yeah, I'd be OK with that - if we're going to reuse the word, it makes
sense to reuse the module to expose the related machinery.

That said, if we do go that way *and* we decide to offer a
coroutine-only backport, I see an offer of contextlib2
co-maintainership in your future ;)

>>>   * If ``coro.cr_local_context`` is an empty ``LocalContext`` object
>>>     that ``coro`` was created with, the interpreter will set
>>>     ``coro.cr_local_context`` to ``None``.
>>
>> I like all the ideas in this section, but this specific point feels a
>> bit weird. Coroutine objects need a second hidden field somewhere to
>> keep track of whether the object they end up with is the same one they
>> were created with?
>
> Yes, I planned to have a second hidden field, as Coroutines will have
> their cr_local_context set to NULL, and that will be their empty LC.
> So a second internal field is needed to disambiguate NULL -- meaning
> an "empty context" and NULL meaning "use outside local context".
>
> I omitted this from the PEP to make it a bit easier to digest, as this
> seemed to be a low-level implementation detail.

Given that the field is writable, I think it makes more sense to just
choose a suitable default, and then rely on other code changing that
default when its not right.

For generators: set it to an empty context by default, have
contextlib.contextmanager (and similar wrapper) clear it
For coroutines: set it to None by default, have async task managers
give top level coroutines their own private context

No hidden flags, no magic value adjustments, just different defaults
for coroutines and generators (including async generators).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia