[Tutor] Yielding from a with block

[Steven D'Aprano]

On Thu, May 28, 2015 at 10:16:00AM +0200, Peter Otten wrote:

> Even if you limit yourself to CPython there is another effect: the order of 
> execution may not meet one's expectations/requirements:

No, wait, forget everything I said in my previous post. This is 
*clearly* a case where our expectations were wrong, and the context 
manager guarantee is working correctly. I was confused because I was 
over-thinking it and seeing something unexpected when in fact it is 
working exactly as promised.


> $ cat with_in_generator.py
> import contextlib
> 
> @contextlib.contextmanager
> def demo():
>     print("before")
>     try:
>         yield
>     finally:
>         print("after")
> 
> def gen(items="abc"):
>     with demo():
>         yield from items
> 
> if __name__ == "__main__":
>     g = gen()
>     for item in g:
>         print(item)
>         if item == "b":
>             break
>     print("bye")

Since you *break* from the for-loop, the generator g never runs to 
completion. Since it is still paused *inside the with block*, naturally 
the context manager __exit__ doesn't run. If it did run, THAT would be a 
violation of the context manager guarantee!

Now that you have broken out of the for-loop, you still have a reference 
to g, and are perfectly entitled to hang on to the reference for a 
while, then iterate over it again, or directly call next(g). Until such 
time as you do, or explicitly call g.close(), the context manager has to 
stay open.
I

Until you do so (or call g.close() to explicitly end it), it is paused 
inside the with block.

There's nothing to see here. The context manager is working correctly, 
and if you expect it to __exit__ while still inside the with block, it 
is your expectations that are wrong.


> (in case you don't spot it: "after" should be printed before "bye")

That's mistaken. Just because you exit from the for-loop, doesn't mean 
the generator is complete. Suppose you wrote a generator like this:

def gen():
    yield 1
    yield 2
    yield 3
    print("closing")

for i in gen():
    if i == 2: break

print("bye")


Would you still expect it to print "closing" before "bye"?


-- 
Steve