[Python-ideas] PEP draft: context variables

Koos Zevenhoven k7hoven at gmail.com
Mon Oct 9 20:37:51 EDT 2017

On Tue, Oct 10, 2017 at 1:55 AM, Yury Selivanov <yselivanov.ml at gmail.com>

> On Mon, Oct 9, 2017 at 4:39 PM, Koos Zevenhoven <k7hoven at gmail.com> wrote:
> > On Mon, Oct 9, 2017 at 6:24 PM, Guido van Rossum <guido at python.org>
> wrote:
> [..]
> >> I'm not sure I agree on the usefulness. Certainly a lot of the
> complexity
> >> of PEP 550 exists just to cater to Nathaniel's desire to influence what
> a
> >> generator sees via the context of the send()/next() call. I'm still not
> sure
> >> that's worth it. In 550 v1 there's no need for chained lookups.
> >
> >
> > We do need some sort of chained lookups, though, at least in terms of
> > semantics. But it is possible to optimize that away in PEP 555.
> You keep using the "optimize away" terminology.  I assume that you
> mean that ContextVar.get() will have a cache (so it does in PEP 550
> btw).  What else do you plan to "optimize away"?  Where's a detailed
> implementation spec?  What you have in the PEP is still vague and
> leaves many important implementation details to the imagination of the
> reader.
​I'm hesitant to call it a cache, because the "cache" sort of automatically
builds itself. I think I'll need to draw a diagram to explain it. The
implementation is somewhat simpler than its explanation. I can go more into
detail regarding the implementation, but I feel that semantics is more
important at this point.

> The fact is that the datastructure choice in PEP 555 is plain weird.
> You want to use a sequence of values to represent a mapping.  And then
> you hand-waved all questions about what will happen in pathological
> cases, saying that "we'll have a cache and applications won't have to
> many context values anyways".

​I don't think I've heard of any pathological cases... What do you mean?​

But your design means that in the worst
> case, the uncached path requires you to potentially traverse all
> values in the context.

​It is in fact possible to implement it in a way that this never happens,
but the best thing might actually be an implementation, where this *almost*
never happens. (Off-topic: It would be kind of cool, if you could do the
same thing with MROs, so OOP method access will speed up​. But that might
be somewhat more difficult to implement, because there are more moving
parts there.)

> Another thing: suppose someone calls
> 'context_var.assign().__enter__()' manually, without calling
> '__exit__()'.  You will have unbound growth of the context values
> stack.

​You can cause unbound growth in PEP 550 too. All you have to do is nest an
unbounded number of generators. In PEP 555, nesting generators doesn't do
anything really, unless you actually assign to context arguments in the
generators.​ Only those who use it will pay.

But seriously, you will always end up in a weird situation if you call an
unbounded number of contextmanager.__enter__() methods without calling
__exit__(). Nothing new about that. But entering a handful of assignment
contexts and leaving them open until a script ends is not the end of the
world. I don't think anyone should do that though.

> You'll say that it's not how the API is supposed to be used,
> and we say that we want to convert things like decimal and numpy to
> use the new mechanism.  That question was also hand-waved by you:
> numpy and decimal will have to come up with new/better APIs to use PEP
> 555.  Well, that's just not good enough.

​What part of my explanation of this are you unhappy with? For instance,
the 12th (I think) email in this thread, which is my response to Nathaniel.
Could you reply to that and tell us your concern?​

> And the key problem is that you still haven't directly highlighted
> differences in semantics between PEP 550 and PEP 555.  This is the
> most annoying part, because almost no one (including me) knows the
> complete answer here.  Maybe you know, but you refuse to include that
> in the PEP for some reason.

I don't refuse to
​. I just haven't prioritized it. But I've probably made the mistake of
mentioning *similarities* between 550 and 555​
​One major difference is that there is no .set(value) in PEP 555, so one
shouldn't try to map PEP 550 uses directly to PEP 555.​

> Some kind of
> > chained-lookup-like thing is inevitable if you want the state not to leak
> > though yields out of the generator:
> No, it's not "inevitable".  In PEP 550 v1, generators captured the
> context when they are created and there was always only one level of
> context.  This means that:
> 1. Context changes in generators aren't visible to the outside world.
> 2. Changes to the context in the outside world are not visible to
> running generators.

​Sure, if you make generators completely isolated from the outside world,
then you can avoid chaining-like things too. But that would just sweep it
under the carpet.

> PEP 550 v1 was the simplest thing possible with a very efficient
> implementation.  It had the following "issues" that led us to v2+
> semantics of chained lookup:
> 1. Refactoring.
> with some_context():
>   for i in gen():
>      pass
> would not be equivalent to:
> g = gen()
> with some_context():
>   for i in g:
>     pass
> ​ ​
​What's the point of this? Moving stuff out of a with statement should not
matter? The whole point of with statements is that it matters whether you
do something inside it or outside it.​

​-- Koos​

> 2. Restricting generators to only see context at the point of their
> creation feels artificial.  We know there are better solutions here
> (albeit more complex) and we try to see if they are worth it.
> 3. Nathaniel't use case in Trio.

+ Koos Zevenhoven + http://twitter.com/k7hoven +
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20171010/e004369e/attachment-0001.html>

More information about the Python-ideas mailing list