[Python-ideas] PEP draft: context variables

[Yury Selivanov]

On Tue, Oct 10, 2017 at 11:26 AM, Koos Zevenhoven <k7hoven at gmail.com> wrote:
> On Tue, Oct 10, 2017 at 5:40 PM, Yury Selivanov <yselivanov.ml at gmail.com>
> wrote:
>>
>> On Tue, Oct 10, 2017 at 8:34 AM, Koos Zevenhoven <k7hoven at gmail.com>
>> wrote:
>> > On Tue, Oct 10, 2017 at 4:22 AM, Yury Selivanov
>> > <yselivanov.ml at gmail.com>
>> > wrote:
>> >>
>> >> On Mon, Oct 9, 2017 at 8:37 PM, Koos Zevenhoven <k7hoven at gmail.com>
>> >> wrote:
>> >> > You can cause unbound growth in PEP 550 too. All you have to do is
>> >> > nest
>> >> > an
>> >> > unbounded number of generators.
>> >>
>> >> You can only nest up to 'sys.get_recursion_limit()' number of
>> >> generators.
>> >>
>> >> With PEP 555 you can do:
>> >>
>> >>   while True:
>> >>     context_var.assign(42).__enter__()
>> >>
>> >
>> > Well, in PEP 550, you can explicitly stack an unbounded number of
>> > LogicalContexts in a while True loop.
>>
>> No, you can't. PEP 550 doesn't have APIs to "stack ... LogicalContexts".
>>
>
> That's ridiculous. Quoting PEP 550: "
> The contextvars.run_with_logical_context(lc: LogicalContext, func, *args,
> **kwargs) function, which runs func with the provided logical context on top
> of the current execution context.

Note that 'run_with_logical_context()' doesn't accept the EC.  It gets
it using the 'get_execution_context()' function, which will squash LCs
if needed.

I say it again: *by design*, PEP 550 APIs do not allow to manually
stack LCs in such a way that an unbound growth of the stack is
possible.

> "
>
>>
>> > Or you can run out of memory using
>> > plain lists even faster:
>> >
>> > l = [42]
>> >
>> > while True:
>> >     l *= 2 # ensure exponential blow-up
>> >
>> > I don't see why your example with context_var.assign(42).__enter__()
>> > would
>> > be any more likely.
>>
>> Of course you can write broken code. The point is that contexts work
>> like scopes/mappings, and it's counter-intuitive that setting a
>> variable with 'cv.assign(..).__enter__()' will break the world.  If a
>> naive user tries to convert their existing decimal-like API to use
>> your PEP, everything would work initially, but then blow up in
>> production.
>>
>
> The docs will tell them what to do. You can pass a context argument down the
> call chain. You don't "set" context arguments! That's why I'm changing to
> "context argument", and I've said this many times now.

I'm saying this the last time: In Python, any context manager should
have an equivalent try..finally form.  Please give us an example, how
we can use PEP 555 APIs with a try..finally block.

By the way, PEP 555 has this, quote:

"""
By default, values assigned inside a generator do not leak through
yields to the code that drives the generator. However, the assignment
contexts entered and left open inside the generator do become
visible outside the generator after the generator has finished with
a StopIteration or another exception:

assi = cvar.assign(new_value)
def genfunc():
    yield
    assi.__enter__():
    yield
"""

Why do you call __enter__() manually in this example?  I thought it's
a strictly prohibited thing in your PEP -- it's unsafe to use it this
way.

Is it only for illustration purposes?  If so, then how "the assignment
contexts entered and left open inside the generator" can even be a
thing in your design?


[..]
>> > * Fall back to thread-local storage if no context argument is present or
>> > if
>> > the Python version does not support context arguments.
>>
>> The last bullet point is the problem.  Everybody is saying to you that
>> it's not acceptable.  It's your choice to ignore that.
>>
>
> Never has anyone told me that that is not acceptable. Please stop that.

The whole idea of PEP 550 was to provide a working alternative to TLS.
So this is clearly not acceptable for PEP 550.

PEP 555 may hand-wave this requirement, but it simply limits the scope
of where it can be useful.  Which in my opinion means that it provides
strictly *less* functionality than PEP 550.

[..]
>> This is plain incorrect. Please read PEP 550v1 before continuing the
>> discussion about it.
>>
>
> I thought you wrote that they are isolated both ways. Maybe there's a
> misunderstanding. I found your "New PEP 550" email in the archives in some
> thread.

PEP 550 has links to all versions of it. You can simply read it there.

> That might be v1, but the figure supposedly explaining this part is
> missing. Whatever. This is not about PEP 550v1 anyway.

This is about you spreading wrong information about PEP 550 (all of
its versions in this case).

Again, in PEP 550:

1. Changes to contexts made in async generators and sync generators do
not leak to the caller.  Changes made in a caller are visible to the
generator.

2. Changes to contexts made in async tasks do not leak to the outer
code or other tasks.  That's assuming async tasks implementation is
tweaked to use 'run_with_execution_context'.  Otherwise, coroutines
work with EC just like functions.

3. Changes to contexts made in OS threads do not leak to other threads.

How's PEP 555 different besides requiring to use a context manager?

>
>>
>> > Also, if you refactor your
>> > generator into subgenerators using `yield from`, the subgenerators will
>> > not
>> > see the context set by the outer generator.
>>
>> Subgenerators see the context changes in the outer generator in all
>> versions of PEP 550.
>>
>> The point you didn't like is that in all versions of PEP 550
>> subgenerators could not leak any context to the outer generator.
>> Please don't confuse these two.
>
>
> That's a different thing. But it's not exactly right: I didn't like the fact
> that some subroutines (functions, coroutines, (async) generators) leak
> context and some don't.

So your PEP is "solving" this by disallowing to simply "set" a
variable without a context manager.  Is this the only difference?

Look, Koos, until you give me a full list of *semantical* differences
between PEP 555 and PEP 550, I'm not going to waste my time on
discussions here.  And I encourage Guido, Nick, and Nathaniel to do
the same.

Yury