Iterate through list two items at a time
Wade Leftwich
wleftwich at gmail.com
Thu Jan 4 08:48:05 EST 2007
Peter Otten wrote:
> Wade Leftwich wrote:
>
> > from itertools import groupby
> >
> > def chunk(it, n=0):
> > if n == 0:
> > return iter([it])
> > def groupfun((x,y)):
> > return int(x/n)
> > grouped = groupby(enumerate(it), groupfun)
> > counted = (y for (x,y) in grouped)
> > return ((z for (y,z) in x) for x in counted)
> >
> >>>> [list(x) for x in chunk(range(10), 3)]
> > [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]
> >
> >>>> [x for x in chunk(range(10), 3)]
> > [<generator object at 0xb7a34e4c>,
> > <generator object at 0xb7a34dac>,
> > <generator object at 0xb7a34d2c>,
> > <generator object at 0xb7a34d6c>]
>
> Note that all but the last of these generators are useless:
>
> >>> chunks = [x for x in chunk(range(10), 3)]
> >>> [list(x) for x in chunks]
> [[], [], [], [9]] # did you expect that?
> In [48]: chunkgen = chunk(range(10), 3)
In [49]: for x in chunkgen:
....: print list(x)
....:
....:
[0, 1, 2]
[3, 4, 5]
[6, 7, 8]
[9]
> Peter
That's an interesting gotcha that I've never run into when using this
function, because in practice I never put the generator returned by
chunk() inside a list comprehension.
In [51]: chunkgen = chunk(range(10), 3)
In [52]: [list(x) for x in chunkgen]
Out[52]: [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]
But, as you pointed out --
In [57]: chunkgen = chunk(range(10), 3)
In [58]: chunks = list(chunkgen)
In [59]: [list(x) for x in chunks]
Out[59]: [[], [], [], [9]]
So apparently when we list(chunkgen), we are exhausting the generators
that are its members. And if we do it again, we get rid of the [9] --
In [60]: [list(x) for x in chunks]
Out[60]: [[], [], [], []]
I'll admit that chunk() is needlessly tricky for most purposes. I wrote
it to accept a really lengthy, possibly unbounded, iterator and write
out 10,000-line files from it, and also to play with Python's new
functional capabilities.
-- Wade
More information about the Python-list
mailing list