[Python-Dev] code blocks using 'for' loops and generators

[Brian Sabbey]

I would like to get some feedback on a proposal to introduce 
smalltalk/ruby-like "code blocks" to python.  Code blocks are, among other 
things, a clean way to solve the "acquire/release" problem [1][2].  This 
proposal also touches on some of the problems PEP 288 [3] deals with.

The best discussion I have been able to find on this topic is in the 
thread of ref. [4].

Since generators, when used with 'for' loops, are so similar to code 
blocks [5], one can imagine two ways to implement code blocks in python: 
(1, parallel) keep them as similar as possible to 'for' loops so that a 
normal user doesn't distinguish the two, or (2, orthogonal) make them so 
much different than 'for' loops that the normal user doesn't notice the 
similarities.  Anything between these extremes will probably be confusing. 
Here I will describe an attempt at (1).

(I) Give generators a __call__ method as an alternative to 'next'. 
Method __call__ should take a single parameter: a function.  Using 
__call__ will cause the generator to start executing normally, but when a 
'yield' is reached, the generator will invoke the function passed to 
__call__ instead of activating the currently implemented 'yield' 
mechanism.

Since __call__ will be an alternative to 'next', it will raise an 
exception if 'next' has already been called and vice-versa.  Also, any 
calls to 'next' will raise an exception if there is a 'yield' in the try 
of a try/finally.  Such yields will no longer trigger a SyntaxError 
because they will not a problem when using __call__.

(II) Have a method of generators, __blockcall__, which will be equal to 
__call__, but will only exist if (1) the generator contains a "try/finally 
try" yield, or (2) the user explicitly defines it, for example, with a 
function decorator (@completion_required would be a descriptive name).

Have 'for' loops use __blockcall__ if it is available, and __iter__ 
otherwise.  Pass to __blockcall__ the block of code in the 'for' loop.

Scope rules for the passed block of code should mimic current 'for' loop 
behavior.  Behavior of 'break' and 'return' should be mimicked, perhaps 
with special exceptions catchable only by the 'for' loop.  Mimicking 
'yield' will be a problem unless/until multi-level yields are allowed. 
(performance and implementation difficulties for all of this?  I don't 
know).

The thunk shouldn't be savable for later use because the 'for' loop will 
no longer be around to deal with 'break' and 'return'.  This means that 
__blockcall__ will not be implementable as a function that takes a 
function as an argument.

(III) Allow 'continue' to pass values to 'yield' (something similar 
previously rejected here [6]).  As far as I know, all other control 
statements that transfer execution to a different frame (yield, return, 
raise) pass values, and I don't see why this case should be any different. 
I do not see such a mechanism as gimmicky;  being able to cleanly pass 
values when changing scope is an inherent part of nearly every programming 
language.

As an example of the syntax I am suggesting, here is something I was 
desiring recently, a generator to open, unpickle, repickle and close a 
file:

def pickled_file(name):
     f = open(name, 'r')
     l yield pickle.load(f)
     f.close()
     f = open(name, 'w')
     pickle.dump(l, f)
     f.close()

The unpickled object is sent to the caller at the yield statement, and the 
modified object is received back at the same statement.  Note the 
suggested 'yield' syntax and the conspicuous absence of '='.  This syntax 
is backwardly compatible with current yield syntax.  Also, this syntax 
does not require yield to appear as a function; it is still clear that 
this is a unique control-flow statement.

This function would be used like this:

for l in pickled_file('greetings.pickle'):
     l.append('hello')
     l.append('howdy')
     continue l

The above code would have the same effect as:

def named(l):
     l.append('hello')
     l.append('howdy')
     return l
pickled_file('greetings.pickle')(named)

(IV)  Allow 'yield' to return no value;  in this case a new keyword, 
'with', will be required instead of an awkward 'for':

"with f():"   instead of   "for in f():"

(V)  For the same reasons as in (III), allow generators to return values. 
These values can be sent with the StopIteration exception if 'next' is 
being used for iteration.  An obvious syntax for receiving these values is 
shown by this example:

with dt = stopwatch():
       f()
       g()
print 'it took', dt, 'seconds'

Although "with stopwatch() result dt:" might not be so bad.


[1] PEP 310 Reliable Acquisition/Release Pairs
 	http://www.python.org/peps/pep-0310.html
[2] PEP 325 Resource-Release Support for Generators
 	http://www.python.org/peps/pep-0325.html
[3] PEP 288 Generators Attributes and Exceptions
 	http://www.python.org/peps/pep-0288.html
[4] http://mail.python.org/pipermail/python-dev/2003-February/032800.html
[5] http://mail.python.org/pipermail/python-dev/2003-February/032826.html
[6] http://mail.python.org/pipermail/python-dev/2002-March/021923.html


-Brian