[Python-Dev] patch: try/finally in generators

[Guido van Rossum]

> > > http://www.python.org/sf/584626
> > > 
> > > This patch removes the limitation of not allowing yield in the
> > > try part of a try/finally. The dealloc function of a generator
> > > checks if the generator is still alive and resumes it one last
> > > time from the return instruction at the end of the code, causing
> > > any try/finally blocks to be triggered. Any exceptions raised
> > > are treated just like exceptions in a __del__ finalizer (printed
> > > and ignored).
> > 
> > I'm not sure I understand what it does.  The return instruction at
> > the end of the code, if I take this literally, isn't enclosed in
> > any try/finally blocks.  So how can this have the desired effect?
> 
> They're on the block stack.  The stack unwind does the rest.

OK.  Your way to find the last return statement gives me the willies
though. :-(

> > Have you verified that Jython can implement these semantics too?
> 
> I don't see why not. The trick of jumping to the end was just my way
> to avoid adding a flag or some magic value to signal to eval_frame
> that it needs to trigger the block stack unwind on ceval.c:2201.
> There must be many other ways to implement this.

Please go to the Jython developers and ask their opinion.
Implementing yield in Java is a bit of a hack, and we've been careful
to make it possible at all.  I don't want to break it.

Of course, since Jython has garbage collection, your finally clause
may be executed later than you had expected it, or not at all!  Are
you sure you want this?  I don't recall all the reasons why this
restriction was added to the PEP, but I believe it wasn't just because
we couldn't figure out how to implement it -- it also had to do with
not being able to explain what exactly the semantics would be.

> > Do you *really* need this?
> 
> I'm a plumber.  I make pipelines by chaining iterators and
> transformations.  My favorite fittings are generator functions and
> closures so I rarely need to actually define a class.  One of my
> generator functions needed to clean up some stuff so I naturally
> used a try/finally block. When the compiler complained I recalled
> that when I first read with excitement about generator functions
> there was a comment there about some arbitrary limitation of yield
> statements in try/finally blocks...
> 
> Anyway, I ended up creating a temporary local object just so I could
> take advantage of its __del__ method for cleanup but I really didn't
> like it.  After a quick look at ceval.c I realized that it would be
> easy to fix this by having the dealloc function simulate a return
> statement just after the yield that was never resumed. So I wrote a
> little patch to remove something that I consider a wart.

There are a few other places that invoke Python code in a dealloc
handler (__del__ invocations in classobject.c and typeobject.c).  They
do a more complicated dance with the reference count.  Can you check
that you are doing the right thing?

I'd also like to get Neil Schemenauer's review of the code, since he
knows best how generators work under the covers.

--Guido van Rossum (home page: http://www.python.org/~guido/)