[Python-ideas] PEP draft: context variables

[Yury Selivanov]

On Mon, Oct 9, 2017 at 8:37 PM, Koos Zevenhoven <k7hoven at gmail.com> wrote:
[..]
>> Another thing: suppose someone calls
>> 'context_var.assign().__enter__()' manually, without calling
>> '__exit__()'.  You will have unbound growth of the context values
>> stack.
>
>
> You can cause unbound growth in PEP 550 too. All you have to do is nest an
> unbounded number of generators.

You can only nest up to 'sys.get_recursion_limit()' number of generators.

With PEP 555 you can do:

  while True:
    context_var.assign(42).__enter__()

> In PEP 555, nesting generators doesn't do
> anything really, unless you actually assign to context arguments in the
> generators. Only those who use it will pay.

Same for 550.  If a generator doesn't set context variables, its LC
will be an empty mapping (or NULL if you want to micro-optimize
things).  Nodes for the chain will come from a freelist. The effective
overhead for generators is a couple operations on pointers, and thus
visible only in microbenchmarks.


>
> But seriously, you will always end up in a weird situation if you call an
> unbounded number of contextmanager.__enter__() methods without calling
> __exit__(). Nothing new about that. But entering a handful of assignment
> contexts and leaving them open until a script ends is not the end of the
> world. I don't think anyone should do that though.
>
>
>>
>> You'll say that it's not how the API is supposed to be used,
>> and we say that we want to convert things like decimal and numpy to
>> use the new mechanism.  That question was also hand-waved by you:
>> numpy and decimal will have to come up with new/better APIs to use PEP
>> 555.  Well, that's just not good enough.
>
>
> What part of my explanation of this are you unhappy with? For instance, the
> 12th (I think) email in this thread, which is my response to Nathaniel.
> Could you reply to that and tell us your concern?

I'm sorry, I'm not going to find some 12th email in some thread.  I
stated in this thread the following: not being able to use PEP 555 to
fix *existing* decimal & numpy APIs is not good enough.  And decimal &
numpy is only one example, there's tons of code out there that can
benefit from its APIs to be fixed to support for async code in Python
3.7.

>
>>
>>
>> And the key problem is that you still haven't directly highlighted
>> differences in semantics between PEP 550 and PEP 555.  This is the
>> most annoying part, because almost no one (including me) knows the
>> complete answer here.  Maybe you know, but you refuse to include that
>> in the PEP for some reason.
>
>
> I don't refuse to
> . I just haven't prioritized it. But I've probably made the mistake of
> mentioning *similarities* between 550 and 555
> .
> One major difference is that there is no .set(value) in PEP 555, so one
> shouldn't try to map PEP 550 uses directly to PEP 555.

This is not a "major difference".  You might feel that it is, but it
is a simple API design choice.  As I illustrated in a few emails
before, as long as users can call 'context_var.assign(..).__enter__()'
manually, your PEP *does* allow to effectively do ".set(value)".

If using a context manager instead of 'set' method is the only
difference you can highlight, then why bother writing a PEP?  The idea
of using context managers to set values is very straightforward and is
easy to be incorporated to PEP 550. In fact, it will be added to the
next version of the PEP (after discussing it with Guido on the lang
summit).

>
>> > Some kind of
>> > chained-lookup-like thing is inevitable if you want the state not to
>> > leak
>> > though yields out of the generator:
>>
>> No, it's not "inevitable".  In PEP 550 v1, generators captured the
>> context when they are created and there was always only one level of
>> context.  This means that:
>>
>> 1. Context changes in generators aren't visible to the outside world.
>> 2. Changes to the context in the outside world are not visible to
>> running generators.
>
>
> Sure, if you make generators completely isolated from the outside world,
> then you can avoid chaining-like things too. But that would just sweep it
> under the carpet.

What do you mean by "just sweep it under the carpet"?  Capturing the
context at the moment of generators creation is a design choice with
some consequences (that I illustrated in my previous email).  There
are cons and pros of doing that.

Yury