[Python-Dev] code blocks using 'for' loops and generators

Josiah Carlson jcarlson at uci.edu
Wed Mar 16 09:39:44 CET 2005

Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Josiah Carlson wrote:
> > Since PEP 310 was already mentioned, can we just say that the discussion
> > can be boiled down to different ways of spelling __enter__/__exit__ from
> > PEP 310?
> It's not quite the same thing. PEP 310 suggests a mechanism
> with a fixed control flow -- do something on entry, do the
> code block, do something on exit. A general block-passing
> mechanism would give complete control to the function as
> to when and how to call the block.

I would like to ask a question.  Does Python want or need Ruby code
blocks?  I ask because that is what is being offered to cure what ails
us.  Don't get me wrong, I'm sure code blocks can solve quite a few
problems (and is used as such in Ruby), but I'm not convinced that it is
the solution for Python.

Any manual on Ruby will invariably discuss code blocks as one of the
most powerful features Ruby has to offer.  Sounds great.  Sounds like a
great big sledgehammer that can be used to do a bunch of things...so
what is currently being proposed as a use for them, and what can't they
do (that would be really nice)?

They are being offered, right now, as a setup and finalization mechanism. 
Essentially a way of allowing people to wrap their own blocks of code in
custom try/finally code, or whatever their minds can think up.  Ok,
__enter__/__exit__ offers that.  What else?

If you were to pass your generator as a code block, then you could
finalize a generator [1], and even raise exceptions in your code block,
but it still wouldn't allow one to pass exceptions into a currently
running generator (a long standing problem such that if we could, then
we would get generator finalization for free [2]).

What else?  Let's dig back into the python-dev archives...

> - synchronized-like, where the block should connect to its environment
> - property-like, where the block should introduce a new scope, and the
>   caller should be able to inspect or control that scope's namespace

The synchronized-like thing is the custom try/finally, aka
__enter__/__exit__ as specified in PEP 310.

The property-like thing was perhaps to be an easier way to generate
properties, which fairly quickly fell to the wayside in discussion,
seemingly because people didn't see a need to add thunks for this.

Later XML DOM parsing came into the discussion, and was quickly
dismissed as being not possible due to the Python parser's limited

Someone else mentioned that thunks could be used to generate a switch
statement, but no elaboration was done, and no PEP was actually written
(switch has also had its own PEP, and even talk of a peephole
optimization for certain if/elif/else blocks to be made into dictionary

So where has all this reading gotten me?  To the point that I believe
previous discussion had concluded that Ruby-style code blocks have
little use in Python.  *shrug*

> However, it's possible that if generators were enhanced
> with some means of allowing yield inside try-finally,
> then generators plus PEP 310 would cover most use cases:
> for-loops and generators when you want to loop, and
> PEP 310 when you don't.
> So rather than coming up with new syntax at this point,
> perhaps it would be better to concentrate on the problem
> of yield inside try-finally. Even if the finally can't
> be guaranteed to run under all conditions, I think it
> would be useful if it could be arranged so that
>    for x in somegenerator():
>      ...
>      raise Blather
>      ...
> would caused any finallies that the generator was suspended
> inside to be executed. Then the semantics would be the
> same as if the for-loop-over-generator were implemented by
> passing a thunk to a function that calls it repeatedly.

PEP 288 using gen.throw() to trigger early finalization, with
try/finally allowed.

 - Josiah

[1] thunk limitations with generator exceptions

def finalize(thunk, seq):

def foo(seq):
    with s=finalize(seq):
        for i in s:
            #do something with i in finalize's namespace...
            yield f(i)
            #raise an exception if desired

for j in foo(seq):
    #do something with j
    #can't raise an exception in foo or finalize's executing code

[2] PEP 288 goodies, with try/finally allowed
(with code duplication, or refactoring, this can be done with

def gen():
        #yields here

a = gen()
for i in a:
    if some condition:
        #stop the generator and call cleanup code

More information about the Python-Dev mailing list