On Tue, Oct 10, 2017 at 1:55 AM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:

On Mon, Oct 9, 2017 at 4:39 PM, Koos Zevenhoven <k7hoven@gmail.com> wrote:
> On Mon, Oct 9, 2017 at 6:24 PM, Guido van Rossum <guido@python.org> wrote:
[..]
>> I'm not sure I agree on the usefulness. Certainly a lot of the complexity
>> of PEP 550 exists just to cater to Nathaniel's desire to influence what a
>> generator sees via the context of the send()/next() call. I'm still not sure
>> that's worth it. In 550 v1 there's no need for chained lookups.
>
>
> We do need some sort of chained lookups, though, at least in terms of
> semantics. But it is possible to optimize that away in PEP 555.

You keep using the "optimize away" terminology. I assume that you
mean that ContextVar.get() will have a cache (so it does in PEP 550
btw). What else do you plan to "optimize away"? Where's a detailed
implementation spec? What you have in the PEP is still vague and
leaves many important implementation details to the imagination of the
reader.

​I'm hesitant to call it a cache, because the "cache" sort of automatically builds itself. I think I'll need to draw a diagram to explain it. The implementation is somewhat simpler than its explanation. I can go more into detail regarding the implementation, but I feel that semantics is more important at this point.

The fact is that the datastructure choice in PEP 555 is plain weird.
You want to use a sequence of values to represent a mapping. And then
you hand-waved all questions about what will happen in pathological
cases, saying that "we'll have a cache and applications won't have to
many context values anyways".

​I don't think I've heard of any pathological cases... What do you mean?​

But your design means that in the worst
case, the uncached path requires you to potentially traverse all
values in the context.

​It is in fact possible to implement it in a way that this never happens, but the best thing might actually be an implementation, where this *almost* never happens. (Off-topic: It would be kind of cool, if you could do the same thing with MROs, so OOP method access will speed up​. But that might be somewhat more difficult to implement, because there are more moving parts there.)

Another thing: suppose someone calls
'context_var.assign().__enter__()' manually, without calling
'__exit__()'. You will have unbound growth of the context values
stack.

​You can cause unbound growth in PEP 550 too. All you have to do is nest an unbounded number of generators. In PEP 555, nesting generators doesn't do anything really, unless you actually assign to context arguments in the generators.​ Only those who use it will pay.

But seriously, you will always end up in a weird situation if you call an unbounded number of contextmanager.__enter__() methods without calling __exit__(). Nothing new about that. But entering a handful of assignment contexts and leaving them open until a script ends is not the end of the world. I don't think anyone should do that though.

You'll say that it's not how the API is supposed to be used,
and we say that we want to convert things like decimal and numpy to
use the new mechanism. That question was also hand-waved by you:
numpy and decimal will have to come up with new/better APIs to use PEP
555. Well, that's just not good enough.

​What part of my explanation of this are you unhappy with? For instance, the 12th (I think) email in this thread, which is my response to Nathaniel. Could you reply to that and tell us your concern?​

And the key problem is that you still haven't directly highlighted
differences in semantics between PEP 550 and PEP 555. This is the
most annoying part, because almost no one (including me) knows the
complete answer here. Maybe you know, but you refuse to include that
in the PEP for some reason.

I don't refuse to

​. I just haven't prioritized it. But I've probably made the mistake of mentioning *similarities* between 550 and 555​

​.

​

​One major difference is that there is no .set(value) in PEP 555, so one shouldn't try to map PEP 550 uses directly to PEP 555.​

> Some kind of
> chained-lookup-like thing is inevitable if you want the state not to leak
> though yields out of the generator:

No, it's not "inevitable". In PEP 550 v1, generators captured the
context when they are created and there was always only one level of
context. This means that:

1. Context changes in generators aren't visible to the outside world.
2. Changes to the context in the outside world are not visible to
running generators.

​Sure, if you make generators completely isolated from the outside world, then you can avoid chaining-like things too. But that would just sweep it under the carpet.

PEP 550 v1 was the simplest thing possible with a very efficient
implementation. It had the following "issues" that led us to v2+
semantics of chained lookup:

1. Refactoring.

with some_context():
for i in gen():
pass

would not be equivalent to:

g = gen()
with some_context():
for i in g:
pass

​What's the point of this? Moving stuff out of a with statement should not matter? The whole point of with statements is that it matters whether you do something inside it or outside it.​ 

​-- Koos​

2. Restricting generators to only see context at the point of their
creation feels artificial. We know there are better solutions here
(albeit more complex) and we try to see if they are worth it.

3. Nathaniel't use case in Trio.

+ Koos Zevenhoven + http://twitter.com/k7hoven +