[Python-ideas] PEP draft: context variables

[Koos Zevenhoven]

On Tue, Oct 10, 2017 at 5:40 PM, Yury Selivanov <yselivanov.ml at gmail.com>
wrote:

> On Tue, Oct 10, 2017 at 8:34 AM, Koos Zevenhoven <k7hoven at gmail.com>
> wrote:
> > On Tue, Oct 10, 2017 at 4:22 AM, Yury Selivanov <yselivanov.ml at gmail.com
> >
> > wrote:
> >>
> >> On Mon, Oct 9, 2017 at 8:37 PM, Koos Zevenhoven <k7hoven at gmail.com>
> wrote:
> >> > You can cause unbound growth in PEP 550 too. All you have to do is
> nest
> >> > an
> >> > unbounded number of generators.
> >>
> >> You can only nest up to 'sys.get_recursion_limit()' number of
> generators.
> >>
> >> With PEP 555 you can do:
> >>
> >>   while True:
> >>     context_var.assign(42).__enter__()
> >>
> >
> > Well, in PEP 550, you can explicitly stack an unbounded number of
> > LogicalContexts in a while True loop.
>
> No, you can't. PEP 550 doesn't have APIs to "stack ... LogicalContexts".
>
>
​​
​That's ridiculous. Quoting PEP 550: "​
The contextvars.run_with_logical_context(lc: LogicalContext, func, *args,
**kwargs) function, which runs func with the provided logical context on
top of the current execution context.
​"​


> > Or you can run out of memory using
> > plain lists even faster:
> >
> > l = [42]
> >
> > while True:
> >     l *= 2 # ensure exponential blow-up
> >
> > I don't see why your example with context_var.assign(42).__enter__()
> would
> > be any more likely.
>
> Of course you can write broken code. The point is that contexts work
> like scopes/mappings, and it's counter-intuitive that setting a
> variable with 'cv.assign(..).__enter__()' will break the world.  If a
> naive user tries to convert their existing decimal-like API to use
> your PEP, everything would work initially, but then blow up in
> production.
>
>
​The docs will tell them what to do. You can pass a context argument down
the call chain. You don't "set" context arguments!​ That's why I'm changing
to "context argument", and I've said this many times now.



> [..]
> > Really, it was my mistake to ever make you think that
> > context_var.assign(42).__enter__() can be compared to .set(42) in PEP
> 550.
> > I'll say it once more: PEP 555 context arguments have no equivalent of
> the
> > PEP-550 .set(..).
>
> Any API exposing a context manager should have an alternative
> try..finally API.  In your case it's
> 'context_var.assign(42).__enter__()'.  'With' statements are sugar in
> Python.  It's unprecedented to design API solely around them.
>
> ​[..]
>

​Yury writes:​


>> >> That question was also hand-waved by you:
> >> >> numpy and decimal will have to come up with new/better APIs to use
> PEP
> >> >> 555.  Well, that's just not good enough.
> >> >
> >> >
>

​Koos writes:​


>> > What part of my explanation of this are you unhappy with? For instance,
> >> > the
> >> > 12th (I think) email in this thread, which is my response to
> Nathaniel.
> >> > Could you reply to that and tell us your concern?
> >>
> >> I'm sorry, I'm not going to find some 12th email in some thread.  I
> >> stated in this thread the following: not being able to use PEP 555 to
> >> fix *existing* decimal & numpy APIs is not good enough.  And decimal &
> >> numpy is only one example, there's tons of code out there that can
> >> benefit from its APIs to be fixed to support for async code in Python
> >> 3.7.
> >>
> >
> > Well, anyone interested can read that 12th email in this thread. In
> short,
> > my recommendation for libraries would be as follows:
> >
> > * If the library does not provide a context manager yet, they should add
> > one, using PEP 555. That will then work nicely in coroutines and
> generators.
> >
> > * If the library does have a context manager, implement it using PEP
> 555. Or
> > to be safe, add a new API function, so behavior in existing async code
> won't
> > change.
> >
> > * If the library needs to support some kind of set_state(..) operation,
> > implement it by getting the state using a PEP 555 context argument and
> > mutating its contents.
> >
> > * Fall back to thread-local storage if no context argument is present or
> if
> > the Python version does not support context arguments.
>
> The last bullet point is the problem.  Everybody is saying to you that
> it's not acceptable.  It's your choice to ignore that.
>
>
​Never has anyone told me that that is not acceptable. Please stop that.

[..]
> >> What do you mean by "just sweep it under the carpet"?  Capturing the
> >> context at the moment of generators creation is a design choice with
> >> some consequences (that I illustrated in my previous email).  There
> >> are cons and pros of doing that.
> >>
> >
> > "Capturing the context at generator creation" and "isolating generators
> > completely" are two different things.
> >
> > I've described pros of the former. The latter has no pros that I'm aware
> of,
> > except if sweeping things under the carpet is considered as one.
> >
> > Yes, the latter works in some use cases, but in others it does not. For
> > instance, if an async framework wants to make some information available
> > throughout the async task. If you isolate generators, then async
> programmers
> > will have to avoid generators, because they don't have access to the
> > information the framework is trying to provide.
>
> This is plain incorrect. Please read PEP 550v1 before continuing the
> discussion about it.
>
>
I thought you wrote that they are isolated both ways. Maybe there's a
misunderstanding. I found your "New PEP 550" email in the archives in some
thread. That might be v1, but the figure supposedly explaining this part is
missing. Whatever. This is not about PEP 550v1 anyway.


> > Also, if you refactor your
> > generator into subgenerators using `yield from`, the subgenerators will
> not
> > see the context set by the outer generator.
>
> Subgenerators see the context changes in the outer generator in all
> versions of PEP 550.
>
> The point you didn't like is that in all versions of PEP 550
> subgenerators could not leak any context to the outer generator.
> Please don't confuse these two.
>

​That's a different thing. But it's not exactly right: I didn't like the
fact that ​some subroutines (functions, coroutines, (async) generators)
leak context and some don't.
​
––Koos


-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20171010/0692b798/attachment.html>