[Python-ideas] Make "yield" inside a with statement a SyntaxError

[Stephen J. Turnbull]

Ken Hilton writes:

 > Can you spot the problem? The "with open(filename)" statement is supposed
 > to ensure that the file object is disposed of properly. However, the "yield
 > f.read()" statement suspends execution within the with block, so if this
 > happened:
 > 
 >     for contents in read_multiple('chunk1', 'chunk2', 'chunk3'):
 >         if contents == 'hello':
 >             break
 > 
 > and the contents of "chunk2" were "hello" then the loop would exit, and
 > "chunk2" would never be closed! Yielding inside a with block, therefore,
 > doesn't make sense and can only lead to obscure bugs.

You have one example of a bad outcome.  That hardly shows that *no*
yield in a with block makes sense.  On the contrary Chris A's
suggestion that a function might yield multiple times within one
context manager seems valid, although it might be bad practice.

There's another possibility.  Context managers have general
initialize-operate-finalize semantics.  True, they *often* have the
simple form "return the value of *expr* bound to *var* at entry, clean
up the contents of *var* at exit".  But I find it easy to imagine that
there are situations where the __enter__ method is complex and the
__exit__ method is a no-op.  In that case the problem you identify
can't happen, and yielding inside such a context manager would be
harmless.  A third possibility is that the __exit__ method has
semantics like the "else" of "while ... else", where the "normal" use
case involves a break, but occasionally the input is exhausted, and
some "default" action needs to be taken.

I grant that I don't have examples of the latter (after all, writing
context managers is a relatively rare activity, and the resource
acquisition-and-release paradigm is prominent).  But Chris's point
about line-oriented generators seems both plausible and common.

His analysis of why the risk is small also seems plausible, so I would
say this problematic case (and it is problematic) is covered by the
"consenting adults" principle.

Steve