[Python-ideas] Yielding through context managers

[Terry Reedy]

On 3/29/2012 8:00 PM, Joshua Bartlett wrote:
> I'd like to propose adding the ability for context managers to catch and
> handle control passing into and out of them via yield and
> generator.send() / generator.next().
>
> For instance,
>
> class cd(object):
>      def __init__(self, path):
>          self.inner_path = path
>
>      def __enter__(self):
>          self.outer_path = os.getcwd()
>          os.chdir(self.inner_path)
>
>      def __exit__(self, exc_type, exc_val, exc_tb):
>          os.chdir(self.outer_path)
>
>      def __yield__(self):
>          self.inner_path = os.getcwd()
>          os.chdir(self.outer_path)
>
>      def __send__(self):
>          self.outer_path = os.getcwd()
>          os.chdir(self.inner_path)
>
> Here __yield__() would be called when control is yielded through the
> with block and __send__() would be called when control is returned via
> .send() or .next(). To maintain compatibility, it would not be an error
> to leave either __yield__ or __send__ undefined.

This strikes me as the wrong solution to the fragility of dubious code. 
The context manager protocol is simple: two special methods. Ditto for 
the iterator protocol. The generator protocol has been complexified; not 
good, but there are benefits and the extra complexity can be ignored. 
But I would be reluctant to complexify the cm protocol. This is aside 
from technical difficulties.

> The rationale for this is that it's sometimes useful for a context
> manager to set global or thread-global state as in the example above,
> but when the code is used in a generator, the author of the generator
> needs to make assumptions about what the calling code is doing. e.g.
>
> def my_generator(path):
>      with cd(path):
>          yield do_something()
>          do_something_else()

Pull the yield out of the with block.

def my_gen(path):
   with cd(path):
     directory = <read directory>
   yield do_something(directory)
   do_else(directory)

or

def my_gen(p):
   with cd(p):
     res = do_something()
   yield res
   with cd(p):
     do_else()

Use same 'result' trick if do_else also yields.

> Even if the author of this generator knows what effect do_something()
> and do_something_else() have on the current working directory, the
> author needs to assume that the caller of the generator isn't touching
> the working directory. For instance, if someone were to create two
> my_generator() generators with different paths and advance them
> alternately, the resulting behaviour could be most unexpected. With the
> proposed change, the context manager would be able to handle this so
> that the author of the generator doesn't need to make these assumptions.

Or make with manipulation of global resources self-contained, as 
suggested above and as intended for with blocks.

-- 
Terry Jan Reedy