[Python-ideas] Batching/grouping function for itertools

[Oscar Benjamin]

On 8 December 2013 12:16, Steven D'Aprano <steve at pearwood.info> wrote:
> On Sun, Dec 08, 2013 at 01:30:56PM +0200, Serhiy Storchaka wrote:
>> 08.12.13 11:25, Steven D'Aprano написав(ла):
>> >In the second case, there is a question about what to do with sequences
>> >that are not a multiple of the window size. Similar to zip(), there are
>> >two things one might do:
>> >
>> >- pad with some given object;
>> >- raise an exception
>>
>> 3) emit last chunk incompleted;
>
> Given a window size of two, and input data [a, b, c], are you suggesting
> a variety that returns this?
>
> (a,b), (c,)

This is the only variant I have ever used. You can see an example use case here:
https://mail.python.org/pipermail//python-ideas/2013-August/022767.html

And this is the implementation I use:

def chunks(iterable, chunksize=100):
    islices = map(islice, repeat(iter(iterable)), repeat(chunksize))
    return takewhile(bool, map(list, islices))


Oscar