[Python-Dev] PEP 343 - Abstract Block Redux

[Nick Coghlan]

Guido van Rossum wrote:
> [Nick Coghlan]
> 
>>That would be good, in my opinion. I updated PEP 3XX to use this idea:
>>http://members.iinet.net.au/~ncoghlan/public/pep-3XX.html
>>
>>With that update (to version 1.6), PEP 3XX is basically PEP 343, but injecting
>>exceptions that occur into the template generator's internal frame instead of
>>invoking next().
> 
> I'm in favor of the general idea, but would like to separate the error
> injection and finalization API for generators into a separate PEP,
> which would then compete with PEP 288 and PEP 325.

Without that it pretty much devolves into the current version of PEP 343, though 
(as far as I can tell, the two PEP's now agree on the semantics of with statements)

> I think the API
> *should* be made public if it is available internally; I don't see any
> implementation reasons why it would simplify the implementation if it
> was only available internally.

If it's internal, we don't need to name it immediately - working out the public 
API can then be decoupled from the ability to use generators to write resource 
managers.

> Here are some issues to resolve in such a PEP.
> 
> - What does _inject_exception() return? It seems that it should raise
> the exception that was passed into it, but I can't find this written
> out explicitly.

That's a good point. The intent was for it to be equivalent to the exception 
being reraised at the point of the last yield. At that point, control flows like 
it would for a call to next() with code inside the generator that looked like:

   yield
   exc_type, value, tb = _passed_in_exception()
   raise exc_type, value, tb

> - What should happen if a generator, in a finally or except clause
> reached from _inject_exception(), executes another yield? I'm tempted
> to make this a legitimate outcome (from the generator's perspective)
> and reserve it for some future statement that implements the looping
> semantics of PEP 340; the *_template decorator's wrapper class however
> should consider it an error, just like other protocol mismatches like
> not yielding the first time or yielding more than once in response to
> next(). Nick's code in fact does all this right, Ijust think it should
> be made explicit.

Yep, that was the intent - you're correct that describing it in the text as well 
would make it clear that this is deliberate.

> - TerminateIteration is a lousy name, since "terminate" means about
> the same as "stop", so there could be legitimate confusion with
> StopIteration. In PEP 340 I used StopIteration for this purpose, but
> someone explained this was a poor choice since existing generators may
> contain code that traps StopIteration for other purposes. Perhaps we
> could use SystemExit for this purpose? Pretty much everybody is
> supposed to let this one pass through since its purpose is only to
> allow cleanup upon program (or thread) exit.

Wouldn't that mean we run the risk of suppressing a *real* SystemExit if it 
occurs while a generator is being finalised?

Perhaps a new exception IteratorExit, which is a subclass of SystemExit. Then 
well-behaved code wouldn't trap it accidentally, and the finalisation code wouldn't?

> - I really don't like reusing __del__ as the method name for any kind
> of destructor; __del__ has all sorts of special semantics (the GC
> treats objects with a __del__ method specially).

I guess if file objects can live without a __del__ method to automatically 
close, generators that require finalisation can survive without it.

> - The all_lines() example jars me. Somehow it bugs me that its caller
> has to remember to care about finalizing it.

The alternative is to have some form of automatic finalisation in for loops, or 
else follow up on PJE's idea of making with statements allow pretty much 
*anything* to be used as VAR1.

Automatic finalisation (like that now described in the Rejected Options section 
of my PEP) makes iterators/generators that manage resources 'just work' (nice) 
but complicates the semantics of generators (bad) and for loops (bad).

The current version of my PEP means you need to know that a particular 
generator/iterator needs finalisation, and rearrange code to cope with that 
(bad), but keeps for loops simple (nice).

PJE's idea about a permissive with statement may actually give the best of both 
worlds - you can *always* use the with statement, and if the supplied object 
doesn't need cleaning up, there's no more overhead than checking for the 
existence of methods in a couple of slots. (See my answer to Phillip on that 
topic for the gory details)

> Maybe it's just not a
> good example;

I was having trouble thinking of a case where it made sense for the generator to 
manage its own resources. I agree all_lines isn't a great example, but its a 
safe bet that things like it will be written once the restriction on yielding 
inside try/finally is lifted.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com