[Python-ideas] PEP draft: context variables

[Yury Selivanov]

On Mon, Oct 9, 2017 at 4:39 PM, Koos Zevenhoven <k7hoven at gmail.com> wrote:
> On Mon, Oct 9, 2017 at 6:24 PM, Guido van Rossum <guido at python.org> wrote:
[..]
>> I'm not sure I agree on the usefulness. Certainly a lot of the complexity
>> of PEP 550 exists just to cater to Nathaniel's desire to influence what a
>> generator sees via the context of the send()/next() call. I'm still not sure
>> that's worth it. In 550 v1 there's no need for chained lookups.
>
>
> We do need some sort of chained lookups, though, at least in terms of
> semantics. But it is possible to optimize that away in PEP 555.

You keep using the "optimize away" terminology.  I assume that you
mean that ContextVar.get() will have a cache (so it does in PEP 550
btw).  What else do you plan to "optimize away"?  Where's a detailed
implementation spec?  What you have in the PEP is still vague and
leaves many important implementation details to the imagination of the
reader.

The fact is that the datastructure choice in PEP 555 is plain weird.
You want to use a sequence of values to represent a mapping.  And then
you hand-waved all questions about what will happen in pathological
cases, saying that "we'll have a cache and applications won't have to
many context values anyways".  But your design means that in the worst
case, the uncached path requires you to potentially traverse all
values in the context.  Another thing: suppose someone calls
'context_var.assign().__enter__()' manually, without calling
'__exit__()'.  You will have unbound growth of the context values
stack.  You'll say that it's not how the API is supposed to be used,
and we say that we want to convert things like decimal and numpy to
use the new mechanism.  That question was also hand-waved by you:
numpy and decimal will have to come up with new/better APIs to use PEP
555.  Well, that's just not good enough.

And the key problem is that you still haven't directly highlighted
differences in semantics between PEP 550 and PEP 555.  This is the
most annoying part, because almost no one (including me) knows the
complete answer here.  Maybe you know, but you refuse to include that
in the PEP for some reason.


> Some kind of
> chained-lookup-like thing is inevitable if you want the state not to leak
> though yields out of the generator:


No, it's not "inevitable".  In PEP 550 v1, generators captured the
context when they are created and there was always only one level of
context.  This means that:

1. Context changes in generators aren't visible to the outside world.
2. Changes to the context in the outside world are not visible to
running generators.

PEP 550 v1 was the simplest thing possible with a very efficient
implementation.  It had the following "issues" that led us to v2+
semantics of chained lookup:

1. Refactoring.

with some_context():
  for i in gen():
     pass

would not be equivalent to:

g = gen()
with some_context():
  for i in g:
    pass

2. Restricting generators to only see context at the point of their
creation feels artificial.  We know there are better solutions here
(albeit more complex) and we try to see if they are worth it.

3. Nathaniel't use case in Trio.

Yury