[Python-ideas] Looking for a "batch" function

Shane Hathaway shane at hathawaymix.org
Sat Jul 17 10:36:28 CEST 2010


Hi all,

An operation I often want in my Python code is some kind of simple batch 
function.  The batch function would take an iterator and return 
same-size batches from it, except the last batch, which could be 
smaller.  Two parameters would be required: the iterator and the size of 
each batch.  Here are some examples of what I would expect this batch 
function to do.

Get batches from a list:

 >>> list(batch([1,2,3,4,5], 2))
[[1, 2], [3, 4], [5]]

Get batches from a string:

 >>> list(batch('one two six ten', 4))
['one ', 'two ', 'six ', 'ten']

Organize a stream of objects into a table:

 >>> list(batch(['Somewhere', 'CA', 90210, 'New York', 'NY', 10001], 3))
[['Somewhere', 'CA', 90210], ['New York', 'NY', 10001]]

My intuition tells me that such a function should exist in Python, but I 
have not found it in the builtin functions, slice operators, or 
itertools.  Did I miss it?

Here is an implementation that satisfies all of the above examples, but 
requires a sliceable sequence as input, not just an iterator:

def batch(input, batch_size):
     while input:
         yield input[:batch_size]
         input = input[batch_size:]

Obviously, I can just include that function in my projects, but I wonder 
if there is some built-in version of it.  If there isn't, maybe there 
should be.

Shane



More information about the Python-ideas mailing list