I've just read through PEP 3156 and I thought I'd resurrect this thread from March. Giving context managers the ability to react to yield and send, and especially to yield from, would allow the eventual introduction of asynchronous locks using PEP 3156 futures. This is one of the open issues listed in the PEP.

Cheers,

J. D. Bartlett.


On 30 March 2012 10:00, Joshua Bartlett <josh@bartletts.id.au> wrote:
I'd like to propose adding the ability for context managers to catch and handle control passing into and out of them via yield and generator.send() / generator.next().

For instance,

class cd(object):
    def __init__(self, path):
        self.inner_path = path

    def __enter__(self):
        self.outer_path = os.getcwd()
        os.chdir(self.inner_path)

    def __exit__(self, exc_type, exc_val, exc_tb):
        os.chdir(self.outer_path)

    def __yield__(self):
        self.inner_path = os.getcwd()
        os.chdir(self.outer_path)

    def __send__(self):
        self.outer_path = os.getcwd()
        os.chdir(self.inner_path)

Here __yield__() would be called when control is yielded through the with block and __send__() would be called when control is returned via .send() or .next(). To maintain compatibility, it would not be an error to leave either __yield__ or __send__ undefined.

The rationale for this is that it's sometimes useful for a context manager to set global or thread-global state as in the example above, but when the code is used in a generator, the author of the generator needs to make assumptions about what the calling code is doing. e.g.

def my_generator(path):
    with cd(path):
        yield do_something()
        do_something_else()

Even if the author of this generator knows what effect do_something() and do_something_else() have on the current working directory, the author needs to assume that the caller of the generator isn't touching the working directory. For instance, if someone were to create two my_generator() generators with different paths and advance them alternately, the resulting behaviour could be most unexpected. With the proposed change, the context manager would be able to handle this so that the author of the generator doesn't need to make these assumptions.

Naturally, nested with blocks would be handled by calling __yield__ from innermost to outermost and __send__ from outermost to innermost.

I rather suspect that if this change were included, someone could come up with a variant of the contextlib.contextmanager decorator to simplify writing generators for this sort of situation.

Cheers,

J. D. Bartlett