[Python-Dev] Scope, not context? (was Re: PEP 550 v3 naming)

[Nick Coghlan]

On 24 August 2017 at 23:52, Barry Warsaw <barry at python.org> wrote:
> Guido van Rossum wrote:
>> On Tue, Aug 22, 2017 at 7:12 PM, Nathaniel Smith <njs at pobox.com> wrote:
>>
>> I worry that that's going to lead more people astray thinking this has
>> something to do with contextlib, which it really doesn't (it's much more
>> closely related to threading.local()).
>
> This is my problem with using "Context" for this PEP.  Although I can't
> keep up with all names being thrown around, it seems to me that in
> Python we already have a well-established meaning for "contexts" --
> context managers, and the protocols they implement as they participate
> in `with` statements.  We have contextlib which reinforces this.  What's
> being proposed in PEP 550 is so far removed from this concept that I
> think it's just going to cause confusion (well, it does in me anyway!).

While I understand the concern, I think context locals and contextlib
are more closely related than folks realise, as one of the main
problems that the PEP is aiming to solve is that with statements (and
hence context managers) *do not work as expected* when their body
includes "yield", "yield from" or "await" .

The reason they don't work reliably is because in the absence of frame
hacks, context managers are currently limited to manipulating process
global and thread local state - there's no current notion of context
local storage that interacts nicely with the frame suspension
keywords. And hence in the absence of any support for context local
state, any state changes made by context managers tend to remain in
effect *even if the frame that entered the context manager gets
suspended*.

Context local state makes it possible to solve that problem, as it
means that encountering one of the frame suspension keywords in the
body of a with statement will implicitly revert all changes made to
context local variables until such time as that particular frame is
resumed, and you have to explicitly opt-in on a per-instance basis to
instead allow those state changes to affect the calling context that's
suspending & resuming the frame.

That said, I can also see Guido's point that the proposed new APIs
have a very different flavour to the existing APIs in contextlib, so
it doesn't necessarily make sense to use that module directly.

Building on the "context locals" naming scheme I've been discussing
elsewhere, one possible name for a dedicated module would be
"contextlocals" giving:

# Read/write access to individual context locals
* context_var = contextlocals.new_context_local(name: str='...')

# Context local storage manipulation
* context = contextlocals.new_context_local_state()
* contextlocals.run_with_context_locals(context: ContextLocalState,
func, *args, **kwargs).
* __context_locals__ attribute on generators and coroutines

# Overall execution context manipulation
* current_ec = contextlocals.get_execution_context()
* new_ec = contextlocals.new_execution_context()
* contextlocals.run_with_execution_context(ec: ExecutionContext, func,
*args, **kwargs).

The remaining connection with "contextlib" would be that
@contextlib.contextmanager (and its async counterpart) would
implicitly clear the __context_locals__ attribute on the underlying
generator instance so that context local state changes made there will
affect the context where the context manager is actually being used.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia