On 14 October 2017 at 08:44, Steve Dower <steve.dower@python.org> wrote:

It's not possible to special case __aenter__ and __aexit__ reliably
(supporting wrappers, decorators, and possible side effects).

Why not? Can you not add a decorator that sets a flag on the code object that means "do not create a new context when called", and then it doesn't matter where the call comes from - these functions will always read and write to the caller's context. That seems generally useful anyway, and then you just say that __aenter__ and __aexit__ are special and always have that flag set.

One example where giving function names implicit semantic significance becomes problematic:

    async def start_transaction(self):

    async def end_transaction(self, *exc_details):

    __aenter__ = start_transaction
    __aexit__ = end_transaction

There *are* ways around that (e.g. type.__new__ implicitly wraps __init_subclass__ with classmethod since it makes no sense as a regular instance method), but then you still run into problems like this:

    async def __aenter__(self):
        return await self.start_transaction()

    async def __aexit__(self, *exc_details):
        return await self.end_transaction(*exc_details)

If coroutines were isolated from their parents by default, then the above method implementations would be broken, even though the exact same invocation pattern works fine for synchronous function calls.

To try and bring this back to synchronous examples that folks may find more intuitive, I figure it's worth framing the question this way: do we want people to reason about context variables like the active context is implicitly linked to the synchronous call stack, or do we want to encourage them to learn to reason about them more like they're a new kind of closure?

The reason I ask that is because there are three "interesting" times in the life of a coroutine or generator:

- definition time (when the def statement runs - this determines the lexical closure)
- instance creation time (when the generator-iterator or coroutine is instantiated)
- execution time (when the frame actually starts running - this determines the runtime call stack)

For synchronous functions, instance creation time and execution time are intrinsically linked, since the execution frame is allocated and executed directly as part of calling the function.

For asynchronous operations, there's more of a question, since actual execution is deferred until you call await or next() - the original synchronous call to the factory function instantiates an object, it doesn't actually *do* anything.

The current position of PEP 550 (which I agree with) is that context variables should default to being closely associated with the active call stack (regardless of whether those calls are regular synchronous ones, or asynchronous ones with await), as this keeps the synchronous and asynchronous semantics of context variables as close to each other as we can feasibly make them.

When implicit isolation takes place, it's either to keep concurrently active logical call stacks isolated from each other (the event loop case), and else to keep context changes from implicitly leaking *up* a stack (the generator case), not to keep context changes from propagating *down* a call stack.

When we do want to prevent downward propagation for some reason, then that's what "run_in_execution_context" is for: deliberate creation of a new concurrently active call stack (similar to running something in another thread to isolate the synchronous call stack).

Don't get me wrong, I'm not opposed to the idea of making it trivial to define "micro tasks" (iterables that perform a context switch to a specified execution context every time they retrieve a new value) that can provide easy execution context isolation without an event loop to manage it, I just think that would be more appropriate as a wrapper API that can be placed around any iterable, rather than being baked in as an intrinsic property of generators.


Nick Coghlan   |   ncoghlan@gmail.com   |   Brisbane, Australia