the usage of 'yield' keyword

[Dave Angel]

Peng Yu wrote:
> http://docs.python.org/reference/simple_stmts.html#grammar-token-yield_stmt
>
> The explanation of yield is not clear to me, as I don't know what a
> generator is. I see the following example using 'yield'. Could
> somebody explain how 'yield' works in this example? Thank you!
>
> def brange(limit):
>   i = 0
>   while i < limit:
>       yield i
>       i += 1
>
>   
Informally, a generator is a convenient syntax for writing an iterator.  
What the above does is match xrange() behavior for single argument, but 
without any limitation on the size or type of the limit parameter.

A little too much detail:  When a function has the keyword "yield" 
within it, the function works much differently than normal functions.  
When it's called as a function, it returns a special object called an 
iterator.  That iterator has a method called next(), which when called 
executes a piece of this function.  It executes the function until it 
encounters a 'yield' statement.  Then the yield value is returned from 
the next() function, but the function is still out-there, suspended.  
All local variables still exist, and it's just waiting for a chance to 
run some more.  Next time next() method is called, the function resumes 
right after the yield, and runs until it gets to another yield (in this 
case the same yield, but with a new value).  That new value is returned 
from the next().  Eventually, the function may 'return' instead of 
'yield', as this one does when the limit value is reached.  At that 
point, it generates a "stop iteration" exception (or some name like 
that).  Now this sounds way too complex.

But, if you study the definition of the for loop, you'll find a 
complementary description.  When you write
    for  item in  iterable:

The iterable is called (I'm blurring the details of when you need 
parens, and when you don't), and next() is called repeatedly, with the 
results of next() being assigned to 'item' until an exception occurs.  
If the exception is 'stop iteration' then the loop terminates normally.

Bottom line is it is easy to write very complicated generators which are 
easy to use in simple for loops.  And although you should also try to 
create an iterator object manually, for the experience, most of the time 
the generator function is much easier/quicker to write, and much easier 
to maintain and debug.



Note that yield can do even more.  I'm trying to describe the common 
usage, which is very convenient.  And it can frequently  be used to take 
a complicated generator and wrap it to make an easier-to-use generator 
that's more specific to a particular purpose.  For example, if os.walk 
is a pain to use, and all you need is a sequence of all the files in a 
directory tree,

def find(root):
   for pdf in os.walk(root, topdown=False):
           for file in pdf[2]:
               yield os.path.join(pdf[0],file)



HTH
DaveA