[Python-ideas] PEP draft: context variables

[Steve Dower]

Nick: “I like Yury's example for this, which is that the following two examples are currently semantically equivalent, and we want to preserve that equivalence:

    with decimal.localcontext() as ctx:
        ctc.prex = 30
        for i in gen():
           pass

    g = gen()
    with decimal.localcontext() as ctx:
        ctc.prex = 30
        for i in g:
          pass”

I’m following this discussion from a distance, but cared enough about this point to chime in without even reading what comes later in the thread. (Hopefully it’s not twenty people making the same point…)

I HATE this example! Looking solely at the code we can see, you are refactoring a function call from inside an *explicit* context manager to outside of it, and assuming the behavior will not change. There’s *absolutely no* logical or semantic reason that these should be equivalent, especially given the obvious alternative of leaving the call within the explicit context. Even moving the function call before the setattr can’t be assumed to not change its behavior – how is moving it outside a with block ever supposed to be safe?

I appreciate the desire to be able to take currently working code using one construct and have it continue working with a different construct, but the burden should be on that library and not the runtime. By that I mean that the parts of decimal that set and read the context should do the extra work to maintain compatibility (e.g. through a globally mutable structure using context variables as a slightly more fine-grained key than thread ID) rather than forcing an otherwise straightforward core runtime feature to jump through hoops to accommodate it.

New users of this functionality very likely won’t assume that TLS is the semantic equivalent, especially when all the examples and naming make it sound like context managers are more related. (I predict people will expect this to behave more like unstated/implicit function arguments and be captured at the same time as other arguments are, but can’t really back that up except with gut-feel. It's certainly a feature that I want for myself more than I want another spelling for TLS…)

Top-posted from my Windows phone

From: Nick Coghlan
Sent: Tuesday, October 10, 2017 5:35
To: Guido van Rossum
Cc: Python-Ideas
Subject: Re: [Python-ideas] PEP draft: context variables

On 10 October 2017 at 01:24, Guido van Rossum <guido at python.org> wrote:
On Sun, Oct 8, 2017 at 11:46 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
On 8 October 2017 at 08:40, Koos Zevenhoven <k7hoven at gmail.com> wrote:
​​I do remember Yury mentioning that the first draft of PEP 550 captured something when the generator function was called. I think I started reading the discussions after that had already been removed, so I don't know exactly what it was. But I doubt that it was *exactly* the above, because PEP 550 uses set and get operations instead of "assignment contexts" like PEP 555 (this one) does. ​​

We didn't forget it, we just don't think it's very useful.


I'm not sure I agree on the usefulness. Certainly a lot of the complexity of PEP 550 exists just to cater to Nathaniel's desire to influence what a generator sees via the context of the send()/next() call. I'm still not sure that's worth it. In 550 v1 there's no need for chained lookups.

The compatibility concern is that we want developers of existing libraries to be able to transparently switch from using thread local storage to context local storage, and the way thread locals interact with generators means that decimal (et al) currently use the thread local state at the time when next() is called, *not* when the generator is created.

I like Yury's example for this, which is that the following two examples are currently semantically equivalent, and we want to preserve that equivalence:

    with decimal.localcontext() as ctx:
        ctc.prex = 30
        for i in gen():
           pass

    g = gen()
    with decimal.localcontext() as ctx:
        ctc.prex = 30
        for i in g:
          pass

The easiest way to maintain that equivalence is to say that even though preventing state changes leaking *out* of generators is considered a desirable change, we see preventing them leaking *in* as a gratuitous backwards compatibility break.

This does mean that *neither* form is semantically equivalent to eager extraction of the generator values before the decimal context is changed, but that's the status quo, and we don't have a compelling justification for changing it.

If folks subsequently decide that they *do* want "capture on creation" or "capture on first iteration" semantics for their generators, those are easy enough to add as wrappers on top of the initial thread-local-compatible base by using the same building blocks as are being added to help event loops manage context snapshots for coroutine execution.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20171010/d3120896/attachment.html>