On 31 March 2017 at 05:22, Nathaniel Smith email@example.com wrote:
On Mon, Mar 20, 2017 at 11:30 AM, Oleg Nesterov firstname.lastname@example.org wrote:
[Aborting "yield" in a "for" loop leaves a sub-generator open, but aborting "yield from" cleans up the sub-generator]
In any case the short answer to your original question is that PEP 342 says that generator finalization calls the generator's close() method, which throws a GeneratorExit into the generator, and PEP 380 says that as a special case, when a GeneratorExit is thrown into a yield from, then this is propagated by calling .close() on the yielded-from iterator (if such a method exists) and then re-raised in the original generator.
I think the Python documentation could be improved regarding this. When I wrote the documentation for coroutine methods https://docs.python.org/3/reference/datamodel.html#coroutine-objects, I included details about the "close" and "throw" methods delegating to inner iterators. I thought I was going to propose similar updates to the generator documentation https://docs.python.org/3/reference/expressions.html#generator-iterator-methods, but it seems I never got around to it. (In the mean time, https://docs.python.org/3/reference/expressions.html#asynchronous-generator-iterator-methods was added, to which this may also be relevant, but that is too complicated for me.)
There is a parallel with another annoyance with Python generator cleanup: https://bugs.python.org/issue28629. There are two ways you can require generators to be used. With a simple generator, you can partially iterate it and then throw it away without any special cleaning up. But with more complex generators that "own" expensive resources, it would be nice to produce a ResourceWarning I you forget to clean them up.
With the "for / yield" case, the sub-generator is not cleaned up, so if a resource-intensive sub-generator has to be cleaned up you have to do it yourself. With "yield from", the cleanup is implicit and unavoidable, which means you can't use it if you want keep the sub-generator alive for later. But the automatic cleanup may be useful in other cases.