[Python-ideas] Looking for a "batch" function

[Shane Hathaway]

On 07/17/2010 02:52 AM, Chris Rebert wrote:
> See the "grouper" recipe in itertools:
> http://docs.python.org/library/itertools.html#recipes
> It does almost exactly what you want:
> grouper(3, 'ABCDEFG', 'x') -->  ['A','B','C'], ['D','E','F'], ['G','x','x']

Interesting, but I have a few concerns with that answer:

- It ignores the type of the container.  If I provide a string as input, 
I expect an iterable of strings as output.

- If I give a batch size of 1000000, grouper() is going to be rather 
inefficient.  Even worse would be to allow users to specify the batch size.

- Since grouper() is not actually in the standard library and it doesn't 
do quite what I need, it's rather unlikely that I'll use it.

Another possible name for this functionality I am describing is 
packetize().  Computers always packetize data for transmission, storage, 
and display to users.  Packetizing seems like such a common need that I 
think it should be built in to Python.

Shane