Generators and nested functions

[Steven Majewski]

On Fri, 25 Jan 2002, Edward C. Jones wrote:

> The following code would be a simple way for doing a useful thing except
> that it doesn't work. Why not? Any easy fixes?
>
> from __future__ import generators
> import os
>
> def walk_dirs(startdir):
>      def visit(arg, dirname, names):
> 	yield dirname
>
>      os.path.walk(startdir, visit, None)
>
> if __name__ == '__main__':
>      for dir in walk_dirs('/some/small/directory'):
> 	print dir
>

Replace the line 'yield dirname' with 'print dirname' and run,
and you'll see what's going on. The visit function doesn't
need to return a value -- it needs to have side effects.
os.path.walk is throwing away any return values, so there's
no way for your "yield" to get thru.

Programming with callback function always seems a bit inside-out
to me, which is one of the reasons I like generators for this
sort of path walking.

For a generator version, try something like:

>>> def files(start):
...     for x in os.listdir(start):
...             x = os.path.join( start, x )
...             yield x
...             if os.path.isdir( x ):
...                     for y in files(x): yield(y)
...
>>> for f in files('.') : print f


With os.path.walk, if you want to trim or filter the files, you need
to do it in the visit function. With a generator version, you can
filter the output of the generator. ( However, trimming branches
that you shouldn't follow is a trickier problem! )

For another method of composing generators see:
<http://groups.google.com/groups?q=generators+group:comp.lang.python+author:Majewski&hl=en&scoring=d&as_drrb=b&as_mind=12&as_minm=1&as_miny=2001&as_maxd=20&as_maxm=12&as_maxy=2001&rnum=1&selm=mailman.1008090197.6318.python-list%40python.org>

which lets you compose filters and generators to do something like:

for x in Files() & isGif & fileOwner('sdm7g') & fileLargerThan(512):
	do_something_with(x)


-- Steve Majewski