[Python-ideas] PEP draft: context variables

Nick Coghlan ncoghlan at gmail.com
Sun Oct 15 03:29:34 EDT 2017

On 15 October 2017 at 15:49, Nathaniel Smith <njs at pobox.com> wrote:

> It's not like this is a new and weird concept in Python either -- e.g.
> when you raise an exception, the relevant 'except' block is determined
> based on where the 'raise' happens (the runtime stack), not where the
> 'raise' was written:
> try:
>     def foo():
>         raise RuntimeError
> except RuntimeError:
>     print("this is not going to execute, because Python doesn't work that
> way")
> foo()

Exactly - this is a better formulation of what I was trying to get at when
I said that we want the semantics of context variables in synchronous code
to reliably align with the semantics of the synchronous call stack as it
appears in an exception traceback.

Attempting a pithy summary of PEP 550's related semantics for use in
explanations to folks that don't care about all the fine details:

    The currently active execution context aligns with the expected flow of
exception handling for any exceptions raised in the code being executed.

And with a bit more detail:

* If the code in question will see the exceptions your code raises, then
your code will also be able to see the context variables that it defined or
* By default, this relationship is symmetrical, such that if your code will
see the exceptions that other code raises as a regular Python exception,
then you will also see the context changes that that code makes.
* However, APIs and language features that enable concurrent code execution
within a single operating system level thread (like event loops, coroutines
and generators) may break that symmetry to avoid context variable
management conflicts between concurrently executing code. This is the key
behavioural difference between context variables (which enable this by
design) and thread local variables (which don't).
* Pretty much everything else in the PEP 550 API design is a lower level
performance optimisation detail to make management of this dynamic state
sharing efficient in event-driven code

Even PEP 550's proposal for how yield would work aligns with that "the
currently active execution context is the inverse of how exceptions will
flow" notion: the idea there is that if a context manager's __exit__ method
wouldn't see an exception raised by a piece of code, then that piece of
code also shouldn't be able to see any context variable changes made by
that context manager's __enter__ method (since the changes may not get
reverted correctly on failure in that case).

Exceptions raised in a for loop body *don't* typically get thrown back into
the body of the generator-iterator, so generator-iterators' context
variable changes should be reverted at their yield points.

By contrast, exceptions raised in a with statement body *do* get thrown
back into the body of a generator decorated with contextlib.contextmanager,
so those context variable changes should *not* be reverted at yield points,
and instead left for __exit__ to handle.

Similarly, coroutines are in the exception handling path for the other
coroutines they call (just like regular functions), so those coroutines
should share an execution context rather than each having their own.

All of that leads to it being specifically APIs that already need to do
special things to account for exception handling flows within a single
thread (e.g. asyncio.gather, asyncio.ensure_future,
contextlib.contextmanager) that are likely to have to put some thought into
how they will impact the active execution context.

Code for which the existing language level exception handling semantics
already work just fine should then also be able to rely on the default
execution context management semantics.


Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20171015/605444ab/attachment.html>

More information about the Python-ideas mailing list