On Tue, 26 May 2009 08:32:58 pm Donald 'Paddy' McCarthy wrote:
Hi, I have a blog entry, http://paddy3118.blogspot.com/2009/05/pipe-fitting-with-python-genera tors.html, in which I define a helper function to get around the problem of reversion in the nesting of generators and the proliferation of nested brackets.
See the blog entry for a fuller treatment, but in essence if you have a series of generators, and want the data to conceptually flow like this:
gen1 -> gen2 -> gen3 -> gen4 ...
You have to write:
With pipe, you would write:
pipe(gen1, gen2, gen3, gen4, ...)
Which you could use like this:
for data in pipe(...): do_something_with_data()
If I use dots for indentation, then maybe the definition will come through to the group:
def pipe(*cmds): ....gen = cmds ....for cmd in cmds[1:]: ....... gen = cmd(gen) ....for x in gen: ....... yield x
The function signature is misleading. It doesn't take a series of generator functions ("cmds"), it takes an initial iterable followed by a series of generator functions.
It seems to me that a cleaner definition would be:
def pipe(iterable, *generators): for gen in generators: iterable = gen(iterable) for x in iterable: yield x
This does seem to be a special case of function composition. If functools grew a compose() function, you could write:
from functools import compose def pipe(it, gens): for x in compose(gens)(it): yield x
Here's an untested definition for compose:
def compose(f, *funcs, **kwargs): if not funcs: raise TypeError('compose() requires at least two functions') if kwargs.keys() not in (, ['doc']):
# I wish Python 2.5 had keyword only args... raise TypeError('bad keyword argument(s)') def function_composition(*args, **kwargs): value = f(*args, **kwargs) for g in funcs: value = g(value) return value function_composition.__doc__ = kwargs.get('doc') return function_composition
which is more complicated than your version, of course, but also more general.
A couple of readers thought that it might be a good tool for itertools to have.
Do you have any other use-cases?
-- Steven D'Aprano