On Tue, 26 May 2009 08:32:58 pm Donald 'Paddy' McCarthy wrote:
Hi, I have a blog entry, http://paddy3118.blogspot.com/2009/05/pipe-fitting-with-python-genera tors.html, in which I define a helper function to get around the problem of reversion in the nesting of generators and the proliferation of nested brackets.
See the blog entry for a fuller treatment, but in essence if you have a series of generators, and want the data to conceptually flow like this:
gen1 -> gen2 -> gen3 -> gen4 ...
You have to write:
...(gen4(gen3(gen2(gen1()))))...
With pipe, you would write:
pipe(gen1, gen2, gen3, gen4, ...)
Which you could use like this:
for data in pipe(...): do_something_with_data()
If I use dots for indentation, then maybe the definition will come through to the group:
def pipe(*cmds): ....gen = cmds[0] ....for cmd in cmds[1:]: ....... gen = cmd(gen) ....for x in gen: ....... yield x
The function signature is misleading. It doesn't take a series of generator functions ("cmds"), it takes an initial iterable followed by a series of generator functions. It seems to me that a cleaner definition would be: def pipe(iterable, *generators): for gen in generators: iterable = gen(iterable) for x in iterable: yield x This does seem to be a special case of function composition. If functools grew a compose() function, you could write: from functools import compose def pipe(it, *gens): for x in compose(*gens)(it): yield x Here's an untested definition for compose: def compose(f, *funcs, **kwargs): if not funcs: raise TypeError('compose() requires at least two functions') if kwargs.keys() not in ([], ['doc']): # I wish Python 2.5 had keyword only args... raise TypeError('bad keyword argument(s)') def function_composition(*args, **kwargs): value = f(*args, **kwargs) for g in funcs: value = g(value) return value function_composition.__doc__ = kwargs.get('doc') return function_composition which is more complicated than your version, of course, but also more general.
A couple of readers thought that it might be a good tool for itertools to have.
Do you have any other use-cases? -- Steven D'Aprano