[Python-ideas] Yield-From: Finalization guarantees
jh at improva.dk
Tue Mar 31 11:44:06 CEST 2009
Greg Ewing wrote:
> Jacob Holm wrote:
>> in most cases this will be code that is breaking the rule about
> > not catching KeyboardInterrupt and SystemExit.
> Not necessarily, it could be doing
> except GeneratorExit:
I said *most* cases, not all. I don't have any proof of this, just a
gut feeling that the majority of generators that convert GeneratorExit
to StopIteration do so because they are using a return in a finally clause.
>> If you use such a generator in a yield-from expression, you will get
>> a RuntimeError('generator ignored GeneratorExit') on close, telling
>> you that something is wrong.
> But it won't be at all clear *what* is wrong or what to
> do about it. The caller is making a perfectly ordinary
> yield-from call, and he's calling what looks to all the
> world like a perfectly well-behaved iterator. Where's
> the mistake?
If this was documented in the PEP, I would say the mistake was in using
such a generator in yield-from that wasn't the final yield. Note that
it is perfectly ok to use such a generator in a yield-from as long as no
outer generator yields afterwards.
> Remember that the generator being called may have been
> written by someone else. The caller may not know anything
> about its internals or be in a position to fix them if
> he did.
Right, that makes it harder to fix the source of the problem.
> > I think that getting a RuntimeError on close is sufficient indication
> > that such a generator should not be used in yield-from.
> But it's a perfectly valid generator by current standards.
> I don't want to declare some existing class of generators
> as being second-class citizens with respect to yield-from,
> especially based on some internal implementation detail
> unknowable to its caller.
I get that.
As I see it we have the following options, listed in my order of preference:
1. Don't throw GeneratorExit to the subiterator but raise it in the
outer generator, and don't explicitly call close. This is the
only version where sharing a subgenerator does not require special
care. It has the problem that it behaves differently in
refcounting and non-refcounting implementations due to the
implicit close that would happen after the yield-from in
refcounting implementations. It also breaks the inlining
principle in the case of throw(GeneratorExit).
2. Do throw GeneratorExit and don't try to reraise it. This is the
version that most closely follows the inlining principle. It has
the problem that generators that convert GeneratorExit to
StopIteration can only be used in a yield-from if none of the
outer generators do a yield afterwards. Breaking this rule gives
a RuntimeError('generator ignored GeneratorExit') on close.
3. Do throw GeneratorExit to the subiterator, and explicitly reraise
it if it was converted to a StopIteration. It has the problem
that it breaks the inlining principle for generators that convert
GeneratorExit to StopIteration.
4. Don't throw GeneratorExit to the subiterator, instead explicitly
call close before raising it in the outer generator. This is the
behavior that #1 would have for non-shared generators in a
refcounting implementation. Same problem as #3 and hides the
GeneratorExit from non-generators.
My guess is that your preference is more like 4, 3, 2, 1. #3 is closest
to what is in the current PEP, and is probably what it meant to say.
(The PEP checks if the thrown exception was GeneratorExit, then does a
bare raise instead of raising the thrown exception).
More information about the Python-ideas