yield_all needed in Python

[Douglas Alan]

While writing a generator, I was just thinking how Python needs a
"yield_all" statement.  With the help of Google, I found a
pre-existing discussion on this from a while back in the Lightweight
Languages mailing list.  I'll repost it here in order to improve the
chances of this enhancement actually happening someday.  The original
poster from the LL mailing list seems mostly concerned with
algorithmic efficiency, while I'm concerned more about making my
programs shorter and easier to read.  The ensuing discussion on the LL
list talks about how yield_all would be somewhat difficult to
implement if you want to get the efficiency gain desired, but I don't
think it would be very difficult to implement if that goal weren't
required, and the goal were limited to just the expressive elegance:

     A Problem with Python's 'yield'

         * To: LL1 Mailing List <address at hidden>
         * Subject: A Problem with Python's 'yield'
         * From: Eric Kidd <address at hidden>
         * Date: 27 May 2003 11:15:20 -0400
         * Organization:
         * Sender: address at hidden

     I'm going to pick on Python here, but only because the example code will
     be short and sweet. :-) I believe several other implementations of
     generators have the same problem.

     Python's generator system, used naively, turns an O(N) tree traversal
     into an O(N log N) tree traversal:

       class Tree:
           def __init__(self, value, left=None, right=None):
               self.value = value
               self.left = left
               self.right = right

           def in_order(self):
               if self.left is not None:
                   for v in self.left.in_order():
                       yield v
               yield self.value
               if self.right is not None:
                   for v in self.right.in_order():
                       yield v

       t=Tree(2, Tree(1), Tree(3))
       for v in yield_bug.t.in_order():
           print v

     This prints:
       1
       2
       3

     Unfortunately, this snippet calls 'yield' 5 times, because the leaf
     values must be yielded twice on their way back up the tree.

     We can shorten the code--and make it run in O(N) time--by adding a new
     keyword to replace the "for v in ...: yield v" pattern:

       def in_order(self):
           if self.left is not None:
               yield_all self.left.in_order():
           yield self.value
           if self.right is not None:
               yield_all self.right.in_order():

     Interestingly enough, this allows you define notions such as
     "tail-recursive generation", and apply the usual bag of
     recursion-optimization techniques.

     Cheers,
     Eric

|>oug