[Python-ideas] Letting context managers react to yields inside their scope

[Nathaniel Smith]

On Wed, Apr 29, 2015 at 2:01 PM, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
> Hi Nathanial,
>
> Sorry for not replying to your last email promptly.
> FWIW I was going to do it later today ;)

Oh well, this way you got a nice clean thread to do it in instead :-)

> My opinion on this subject (and I've implemented lots
> of local-context kind of objects in different frameworks) is
> that *just* inserting some kind of suspend/resume points
> before/after yields does not work.

Before I get into the details of your message, I just want to
emphasize-- there are clearly cases where inserting some kind of
suspend/resume points before/after yields is not just a sufficient
solution, but actually the only possible solution.

Consider a streaming data processing pipeline that uses a generator to
load data from disk in chunks, and then a series of generators to
process, split, filter, etc. this data before passing it on to the
next generator in the stack. For certain problems this is 100% the
best way to do it, generators are amazing. But right now you kinda...
just cannot use certain kinds of 'with' blocks in such code without
all kinds of weirdness ensuing. And coming up with fancy APIs for
event loops to use won't help, because there are no event loops here
at all. The problem here isn't caused by event loops, it's caused by
the way adding coroutines to Python created a dissociation between a
block's dynamic-scope-in-terms-of-lifetime and its
dynamic-scope-in-terms-of-call-chain.

Obviously there may be other cases where you also need something else
on top of this proposal (though I'm not 100% convinced... see below).
But logically speaking, even if you're right that this proposal is not
sufficient, that doesn't imply that it's not necessary :-).

> Why:
>
> 1. Greenlets, gevent, eventlet, stackless python: they do
> not have 'yield'. Context switches are invisible to the
> interpreter.  And those frameworks are popular.  You want
> numpy.errstate/decimal.localcontext to work there too.

First, this is the only argument you're giving for "why" yield
callbacks are insufficient, right? (I'm a little confused by your
formatting and want to make sure I'm not missing anything.) Also, just
to check, the only actual library that we really have to worry about
here is greenlets, right, b/c gevent and eventlet use greenlets to do
the heavy lifting, and stackless is a whole-interpreter fork so for
them context switches are visible to the interpreter?

Now, as to the actual point, I have a few reactions:

a) The curmudgeonly reaction: Well, yes, the problem with those
frameworks is that switches are invisible to both the interpreter and
to user code, which makes it more difficult to manage shared state of
all kinds, not just the very specific pseudo-dynamic-scoping state
that we're talking about here. If you want a programming model where
you can actually keep track of shared state, then maybe you should use
something like asyncio instead :-).

b) Okay, fine, if you want it to just work anyway: any implementation
of this proposal will require that the interpreter have some way to
walk the "with block stack" (so to speak) and call the __suspend__ and
__resume__ methods on demand. greenlets could also walk up that stack
when suspending execution. (This would require the library to do a bit
of nasty mucking about in the CPython internals, but c'mon, this is
greenlets we're talking about. Compared to the other stuff it does
this would hardly even register.) The one caveat here is that if we
want this to be possible then it would require we skip statically
optimizing out __suspend__/__resume__ checking in regular functions.

c) Backup plan: greenlets could provide an API like:
greenlets.register_suspend_resume_callbacks(),
greenlets.unregister_suspend_resume_callbacks(), and context managers
that wanted to handle both kinds of suspensions could define
__suspend__/__resume__ and then do

    def __enter__(self):
        greenlets.register_suspend_resume_callbacks(self.__suspend__,
self.__resume__)
        # ... actual code ...
    def __exit__(self):
        greenlets.unregister_suspend_resume_callbacks(self.__suspend__,
self.__resume__)
        # ... actual code ...

and everything would AFAICT just work. Kinda annoying, but I'm pretty
sure that there is and will always be exactly one greenlets library
across the whole Python ecosystem, so it's not much of a limitation.

> 2. I'm curious how this will be implemented and what's
> the performance impact.

I am also curious about the performance impact :-). I don't see any
reason to expect it to be large, but one never knows for sure until
one has an implementation.

As for implementation, the most straightforward way I think would be
to (1) add a variant of the SETUP_WITH bytecode that also looks up
__suspend__ and __resume__ and pushes them into the value stack and
then uses a unique b_type for its block stack entry, (2) add a variant
of YIELD_VALUE that walks the block stack looking for this unique
b_type and calls __suspend__ when found (b_level points to where to
find it on the value stack), plus maybe as an optimization setting a
flag on the frame noting that __resume__ will be needed, (3) tweak the
resumption code so that it checks for this flag and if found walks the
block stack calling __resume__, (4) add a variant of WITH_CLEANUP that
knows how to clean up this more complex stack, (5) make sure that the
unwind code knows to treat this new block type the same way it treats
SETUP_FINALLY blocks.

That's based on like 30 minutes of thought starting from zero
knowledge of how this part of ceval.c works, so knowing ceval.c there
are certainly a few dragons lurking that I've missed. But as an
initial high-level sketch does it answer your question?

> 3. I think that mechanism should be more generic than
> just 'with' staments. What if you want to have a decorator
> that applies some context?

I don't understand the problem here. A coroutine decorator already
knows when the wrapped coroutine is suspended/resumed, so there's
nothing to fix -- basically for decorators this proposal is already
implemented :-).

> What if you can't write it as
> a generator with @contextlib.contextmanager?

Here I just don't understand the sentence :-). Can you rephrase?

Answering one possible question you might have intended:
@contextmanager indeed is not useful for defining a context manager
that has __suspend__ and __resume__ callbacks, but this is just a
convenience utility. My suspicion is that these kinds of context
managers are already sufficiently rare and tricky to write that having
special utilities to make it easier is not that important, but if
someone comes up with a nice API for doing so then cool, we could put
that in contextlib.

> What I would propose:
>
> I think that the right approach here would be to have a
> standard protocol; something similar to how we defined
> WSGI -- it doesn't require any changes in the interpreter,
> it is a protocol.
>
> For instance, we could have a module in the stdlib, with a
> class Context:
>
>    class Context:
>       @classmethod
>       def __suspend_contexts(cls):
>          for child in cls.__subclasses__():
>             child.__suspend_context()
>
>       @classmethod
> def __resume_contexts(cls): ..
>
>
> To inform context objects that the context is about to change,
> asyncio event loop would call Context.__suspend_contexts()

I can't see how to usefully implement, well, anything, using this
interface as given. Maybe you mean something like

class Context:
    @classmethod
    def switch_contexts(cls, old_ctx_id, new_ctx_id):
        for child in cls.__subclasses__():
            child.switch_contexts(old_ctx_id, new_ctx_id)

    @classmethod
    def destroy_context(cls, ctx_id):
        for child in cls.__subclasses__():
            child.destroy_context(ctx_id)

?

Or a better approach that avoids having to generate and keep track of
arbitrary identifiers would be:

class Context:
    @classmethod
    def new_context(cls):
        return {subcls: subcls.new_context() for subcls in cls.__subclasses__()}

    @classmethod
    def get_context(cls):
        return {subcls: subcls.get_context() for subcls in cls.__subclasses__()}

    @classmethod
    def set_context(cls, context):
        for subcls in cls.__subclasses__():
            subcls.set_context(context[subcls])

And then you could use it like:

class myframework_context:
    def __enter__(self):
        self.old_context = Context.get_context()
        self.context = Context.new_context()
        Context.set_context(self.context)

    def __suspend__(self):
        Context.set_context(self.old_context)

    def __resume__(self):
        Context.set_context(self.context)

    def __exit__(self):
        Context.set_context(self.old_context)

def myframework_run_task(coro):
    with myframework_context:
        yield from coro

and voila, works even if you have multiple event loops calling into
each other :-)

Okay, I kid, but my point is that these different approaches really
are not very different. The big advantage of this approach over ones
that involve doing stuff like main_loop.get_dynamic_namespace() is
that it lets each piece of state be stored for regular usage in
whatever form makes sense -- e.g. numpy's errstate can live in a plain
old TLS variable and be accessed quickly from there, and we only have
to do expensive stuff at context switch time. But this also means that
your context switches are now notionally identical to my context
switches -- they call some suspend/resume callbacks and let each piece
of state push/pop itself as appropriate.

The real difference is just that you assume that every Python program
has exactly one piece of manager code somewhere that knows about all
context switches, so you have it do the calls, and I assume that there
may be many context switch points within a single program (because,
well, Python has dedicated syntax for such context switches built into
the language itself), so I think Python itself should do the calls at
each appropriate place.

> I think that it's up to the event loop/library/framework
> to manage context switches properly.  In asyncio, for instance,
> you can attach callbacks to Futures. I think it would be
> great if such callbacks are executed in the right context.

Again, these approaches aren't necessarily contradictory. For
something like a request id to be used in logging, which is tightly
coupled to the flow of data through your async application, what you
say makes perfect sense. For something like the decimal context, it
feels pretty weird to me.

> I would also suggest to think about a universal context --
> the one that works both for asyncio coroutines and
> threadpools they might use.

I'm afraid I have no idea what this means either.

-n

-- 
Nathaniel J. Smith -- http://vorpus.org