Greg Ewing wrote:
Jacob Holm wrote:
in most cases this will be code that is breaking the rule about not catching KeyboardInterrupt and SystemExit.
Not necessarily, it could be doing
except GeneratorExit: return
I said *most* cases, not all. I don't have any proof of this, just a gut feeling that the majority of generators that convert GeneratorExit to StopIteration do so because they are using a return in a finally clause.
If you use such a generator in a yield-from expression, you will get a RuntimeError('generator ignored GeneratorExit') on close, telling you that something is wrong.
But it won't be at all clear *what* is wrong or what to do about it. The caller is making a perfectly ordinary yield-from call, and he's calling what looks to all the world like a perfectly well-behaved iterator. Where's the mistake?
If this was documented in the PEP, I would say the mistake was in using such a generator in yield-from that wasn't the final yield. Note that it is perfectly ok to use such a generator in a yield-from as long as no outer generator yields afterwards.
Remember that the generator being called may have been written by someone else. The caller may not know anything about its internals or be in a position to fix them if he did.
Right, that makes it harder to fix the source of the problem.
I think that getting a RuntimeError on close is sufficient indication that such a generator should not be used in yield-from.
But it's a perfectly valid generator by current standards. I don't want to declare some existing class of generators as being second-class citizens with respect to yield-from, especially based on some internal implementation detail unknowable to its caller.
I get that. As I see it we have the following options, listed in my order of preference: 1. Don't throw GeneratorExit to the subiterator but raise it in the outer generator, and don't explicitly call close. This is the only version where sharing a subgenerator does not require special care. It has the problem that it behaves differently in refcounting and non-refcounting implementations due to the implicit close that would happen after the yield-from in refcounting implementations. It also breaks the inlining principle in the case of throw(GeneratorExit). 2. Do throw GeneratorExit and don't try to reraise it. This is the version that most closely follows the inlining principle. It has the problem that generators that convert GeneratorExit to StopIteration can only be used in a yield-from if none of the outer generators do a yield afterwards. Breaking this rule gives a RuntimeError('generator ignored GeneratorExit') on close. 3. Do throw GeneratorExit to the subiterator, and explicitly reraise it if it was converted to a StopIteration. It has the problem that it breaks the inlining principle for generators that convert GeneratorExit to StopIteration. 4. Don't throw GeneratorExit to the subiterator, instead explicitly call close before raising it in the outer generator. This is the behavior that #1 would have for non-shared generators in a refcounting implementation. Same problem as #3 and hides the GeneratorExit from non-generators. My guess is that your preference is more like 4, 3, 2, 1. #3 is closest to what is in the current PEP, and is probably what it meant to say. (The PEP checks if the thrown exception was GeneratorExit, then does a bare raise instead of raising the thrown exception). - Jacob