[Python-Dev] Re: anonymous blocks

Guido van Rossum gvanrossum at gmail.com
Mon Apr 25 01:57:14 CEST 2005

After reading a lot of contributions (though perhaps not all -- this
thread seems to bifurcate every time someone has a new idea :-) I'm
back to liking yield for the PEP 310 use case. I think maybe it was
Doug Landauer's post mentioning Beta, plus scanning some more examples
of using yield in Ruby. Jim Jewett's post on defmacro also helped, as
did Nick Coghlan's post explaining why he prefers 'with' for PEP 310
and a bare expression for the 'with' feature from Pascal (and other
languages :-).

It seems that the same argument that explains why generators are so
good for defining iterators, also applies to the PEP 310 use case:
it's just much more natural to write

    def with_file(filename):
        f = open(filename)
            yield f

than having to write a class with __entry__ and __exit__ and
__except__ methods (I've lost track of the exact proposal at this

At the same time, having to use it as follows:

    for f in with_file(filename):
        for line in f:
            print process(line)

is really ugly, so we need new syntax, which also helps with keeping
'for' semantically backwards compatible. So let's use 'with', and then
the using code becomes again this:

    with f = with_file(filename):
        for line in f:
            print process(line)

Now let me propose a strawman for the translation of the latter into
existing semantics. Let's take the generic case:

    with VAR = EXPR:

This would translate to the following code:

    it = EXPR
    err = None
    while True:
            if err is None:
                VAR = it.next()
                VAR = it.next_ex(err)
        except StopIteration:
            err = None
        except Exception, err: # Pretend "except Exception:" == "except:"
            if not hasattr(it, "next_ex"):

(The variables 'it' and 'err' are not user-visible variables, they are
internal to the translation.)

This looks slightly awkward because of backward compatibility; what I
really want is just this:

    it = EXPR
    err = None
    while True:
            VAR = it.next(err)
        except StopIteration:
            err = None
        except Exception, err: # Pretend "except Exception:" == "except:"

but for backwards compatibility with the existing argument-less next()
API I'm introducing a new iterator API next_ex() which takes an
exception argument.  If that argument is None, it should behave just
like next().  Otherwise, if the iterator is a generator, this will
raised that exception in the generator's frame (at the point of the
suspended yield).  If the iterator is something else, the something
else is free to do whatever it likes; if it doesn't want to do
anything, it can just re-raise the exception.

Also note that, unlike the for-loop translation, this does *not*
invoke iter() on the result of EXPR; that's debatable but given that
the most common use case should not be an alternate looping syntax
(even though it *is* technically a loop) but a more general "macro
statement expansion", I think we can expect EXPR to produce a value
that is already an iterator (rather than merely an interable).

Finally, I think it would be cool if the generator could trap
occurrences of break, continue and return occurring in BODY.  We could
introduce a new class of exceptions for these, named ControlFlow, and
(only in the body of a with statement), break would raise BreakFlow,
continue would raise ContinueFlow, and return EXPR would raise
ReturnFlow(EXPR) (EXPR defaulting to None of course).

So a block could return a value to the generator using a return
statement; the generator can catch this by catching ReturnFlow.
(Syntactic sugar could be "VAR = yield ..." like in Ruby.)

With a little extra magic we could also get the behavior that if the
generator doesn't handle ControlFlow exceptions but re-raises them,
they would affect the code containing the with statement; this means
that the generator can decide whether return, break and continue are
handled locally or passed through to the containing block.

Note that EXPR doesn't have to return a generator; it could be any
object that implements next() and next_ex().  (We could also require
next_ex() or even next() with an argument; perhaps this is better.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

More information about the Python-Dev mailing list